[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:19 dist-test-slave-zpss krb5kdc[18168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:19 dist-test-slave-zpss krb5kdc[18168](info): set up 2 sockets
Jun 20 23:03:19 dist-test-slave-zpss krb5kdc[18168](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 23:03:20.106299 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:20.218104 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45153
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:45153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:20.295197 18181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:20.295413 18181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:20.295451 18181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:20.296576 18181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:20.296634 18181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:20.296658 18181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:20.296685 18181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:20.296707 18181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:20.298098 18181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:45153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:20.298353 18181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:20.298557 18181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:20.303046 18187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:20.303059 18186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:20.303064 18189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:20.303325 18181 server_base.cc:1034] running on GCE node
I20260620 23:03:20.303571 18181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:20.303838 18181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:20.304972 18181 hybrid_clock.cc:648] HybridClock initialized: now 1781996600304956 us; error 41 us; skew 500 ppm
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:20.307400 18181 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:20.307819 18181 webserver.cc:460] Webserver started at http://127.17.188.126:38211/ using document root <none> and password file <none>
I20260620 23:03:20.308043 18181 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:20.308108 18181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:20.308259 18181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:20.309204 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "b1d155db80d647c1958dce785b9cfc40"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "1fdd15f2feac35aefa6dc990218de8f0"
server_key_iv: "55cca3f7c4a3f3557714ba7828b55579"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.309623 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "b1d155db80d647c1958dce785b9cfc40"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "1fdd15f2feac35aefa6dc990218de8f0"
server_key_iv: "55cca3f7c4a3f3557714ba7828b55579"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.311022 18181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:20.311764 18196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.311921 18181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:20.312016 18181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "b1d155db80d647c1958dce785b9cfc40"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "1fdd15f2feac35aefa6dc990218de8f0"
server_key_iv: "55cca3f7c4a3f3557714ba7828b55579"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.312093 18181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:20.330506 18181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:20.332840 18181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:20.332993 18181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:20.336846 18181 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:45153
I20260620 23:03:20.336882 18248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:45153 every 8 connection(s)
I20260620 23:03:20.337231 18181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:20.338068 18249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:20.340171 18249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40: Bootstrap starting.
I20260620 23:03:20.340682 18249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:20.340896 18249 log.cc:824] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:20.341572 18249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40: No bootstrap required, opened a new log
I20260620 23:03:20.342036 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 18181
I20260620 23:03:20.342164 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:20.342749 18161 external_mini_cluster.cc:1428] Setting key 35f73fd8d4861f84d047e3ba0ba7c2da
I20260620 23:03:20.343052 18249 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d155db80d647c1958dce785b9cfc40" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45153 } }
I20260620 23:03:20.343176 18249 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:20.343197 18249 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d155db80d647c1958dce785b9cfc40, State: Initialized, Role: FOLLOWER
I20260620 23:03:20.343374 18249 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d155db80d647c1958dce785b9cfc40" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45153 } }
I20260620 23:03:20.343464 18249 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:20.343497 18249 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:20.343741 18249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:20.344605 18249 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d155db80d647c1958dce785b9cfc40" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45153 } }
I20260620 23:03:20.344712 18249 leader_election.cc:302] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1d155db80d647c1958dce785b9cfc40; no voters:
I20260620 23:03:20.344874 18249 leader_election.cc:288] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:20.344897 18254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:20.345000 18254 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [term 1 LEADER]: Becoming Leader. State: Replica: b1d155db80d647c1958dce785b9cfc40, State: Running, Role: LEADER
I20260620 23:03:20.345077 18254 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d155db80d647c1958dce785b9cfc40" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45153 } }
I20260620 23:03:20.345126 18249 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:20.345422 18254 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1d155db80d647c1958dce785b9cfc40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d155db80d647c1958dce785b9cfc40" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45153 } } }
I20260620 23:03:20.345568 18254 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:20.345759 18254 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1d155db80d647c1958dce785b9cfc40. Latest consensus state: current_term: 1 leader_uuid: "b1d155db80d647c1958dce785b9cfc40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d155db80d647c1958dce785b9cfc40" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45153 } } }
I20260620 23:03:20.345844 18254 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:20.346236 18262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:20.346845 18262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:20.349702 18262 catalog_manager.cc:1347] Generated new cluster ID: 40c2249c0c8946d88e87e55df4196617
I20260620 23:03:20.349757 18262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:20.359519 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:20.343581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37890 (local address 127.17.188.126:45153)
0620 23:03:20.343934 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:20.343944 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:20.344016 (+ 72us) server_negotiation.cc:407] Connection header received
0620 23:03:20.346105 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:20.346112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:20.346167 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:20.346310 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:20.348379 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.349241 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.350730 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.351333 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.351443 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:20.355604 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:20.355608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:20.355608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:20.355619 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:20.357554 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.358026 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.358029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.358029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.358134 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.358403 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.358404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.358404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.358719 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:20.358878 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:20.359155 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 23:03:20.359297 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":225,"threads_started":1}
I20260620 23:03:20.360800 18262 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:20.361246 18262 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:20.366541 18262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1d155db80d647c1958dce785b9cfc40: Generated new TSK 0
I20260620 23:03:20.366700 18262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:03:20.395148 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45153
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:20.487143 18275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:20.487336 18275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:20.487368 18275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:20.488457 18275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:20.488495 18275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:20.488520 18275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:20.490049 18275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45153
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:20.490263 18275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:20.490495 18275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:20.495234 18283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:20.495316 18281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:20.495222 18280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:20.495474 18275 server_base.cc:1034] running on GCE node
I20260620 23:03:20.495661 18275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:20.495935 18275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:20.497085 18275 hybrid_clock.cc:648] HybridClock initialized: now 1781996600497075 us; error 27 us; skew 500 ppm
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:20.499562 18275 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:03:20.499987 18275 webserver.cc:460] Webserver started at http://127.17.188.65:32809/ using document root <none> and password file <none>
I20260620 23:03:20.500159 18275 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:20.500195 18275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:20.500277 18275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:20.501238 18275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "71b04f5da3c84e5681f6aefb2312d558"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "af70d64433f35e0d60a44c543f030cfb"
server_key_iv: "3762d1e4757f651d5715edee7187d797"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.501645 18275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "71b04f5da3c84e5681f6aefb2312d558"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "af70d64433f35e0d60a44c543f030cfb"
server_key_iv: "3762d1e4757f651d5715edee7187d797"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.502859 18275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:20.503477 18290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.503643 18275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.503736 18275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "71b04f5da3c84e5681f6aefb2312d558"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "af70d64433f35e0d60a44c543f030cfb"
server_key_iv: "3762d1e4757f651d5715edee7187d797"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.503818 18275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:20.536989 18275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:20.539376 18275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:20.539551 18275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:20.539793 18275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:20.540158 18275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:20.540208 18275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.540242 18275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:20.540287 18275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.546322 18275 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:34967
I20260620 23:03:20.546365 18403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:34967 every 8 connection(s)
I20260620 23:03:20.546797 18275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:20.550185 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 18275
I20260620 23:03:20.550279 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:20.550422 18161 external_mini_cluster.cc:1428] Setting key 855afc6e19d974274a8e667e152926d1
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:20.556157 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:20.547444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:49601 (local address 127.17.188.126:45153)
0620 23:03:20.547500 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:20.547502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:20.547693 (+ 191us) server_negotiation.cc:407] Connection header received
0620 23:03:20.548143 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:20.548144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:20.548158 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:20.548208 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:20.549064 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.549551 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.550566 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.550824 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.550891 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:20.553353 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:20.553355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:20.553355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:20.553364 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:20.554718 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.555236 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.555237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.555237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.555290 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.555556 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.555557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.555557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.555647 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:20.555694 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:20.556034 (+ 340us) server_negotiation.cc:299] Negotiation successful
0620 23:03:20.556097 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:03:20.556464 18406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:20.547447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45153 (local address 127.17.188.65:49601)
0620 23:03:20.547593 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:20.547625 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:20.548063 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:20.548257 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:20.548260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:20.548810 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:20.548993 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:20.548997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.549620 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.549621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:20.550464 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:20.550467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.550904 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.550905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:20.551053 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:20.551573 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:20.551589 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:20.553226 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:20.554796 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:20.554803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:20.554805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:20.555142 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:20.555349 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:20.555350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:20.555350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:20.555491 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:20.555710 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:20.555716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:20.555894 (+ 178us) client_negotiation.cc:769] Sending connection context
0620 23:03:20.556022 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 23:03:20.556192 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260620 23:03:20.557016 18404 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45153
I20260620 23:03:20.557114 18404 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:20.557354 18404 heartbeater.cc:511] Master 127.17.188.126:45153 requested a full tablet report, sending...
I20260620 23:03:20.557895 18213 ts_manager.cc:194] Registered new tserver with Master: 71b04f5da3c84e5681f6aefb2312d558 (127.17.188.65:34967)
I20260620 23:03:20.558485 18213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:49601
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:03:20.574402 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45153
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:20.649814 18409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:20.650019 18409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:20.650058 18409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:20.651186 18409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:20.651247 18409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:20.651290 18409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:20.652670 18409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45153
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:20.652937 18409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:20.653141 18409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:20.657989 18415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:20.657981 18417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:20.657979 18414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:20.658087 18409 server_base.cc:1034] running on GCE node
I20260620 23:03:20.658342 18409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:20.658591 18409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:20.659749 18409 hybrid_clock.cc:648] HybridClock initialized: now 1781996600659730 us; error 30 us; skew 500 ppm
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:20.662204 18409 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:03:20.662606 18409 webserver.cc:460] Webserver started at http://127.17.188.66:36297/ using document root <none> and password file <none>
I20260620 23:03:20.662825 18409 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:20.662875 18409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:20.662959 18409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:20.663904 18409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "bafe6cb940b04fdead8513e86b592f28"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "63225c699cbbb0940afe473785545894"
server_key_iv: "fd978b7fc2e32fa46588b7fdf7344432"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.664192 18409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "bafe6cb940b04fdead8513e86b592f28"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "63225c699cbbb0940afe473785545894"
server_key_iv: "fd978b7fc2e32fa46588b7fdf7344432"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.665467 18409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:03:20.666100 18424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.666253 18409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:20.666344 18409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "bafe6cb940b04fdead8513e86b592f28"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "63225c699cbbb0940afe473785545894"
server_key_iv: "fd978b7fc2e32fa46588b7fdf7344432"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.666421 18409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:20.683607 18409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:20.685758 18409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:20.685909 18409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:20.686106 18409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:20.686380 18409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:20.686431 18409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.686470 18409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:20.686494 18409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.692034 18409 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:46393
I20260620 23:03:20.692065 18537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:46393 every 8 connection(s)
I20260620 23:03:20.692346 18409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:20.698284 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 18409
I20260620 23:03:20.698372 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:20.698560 18161 external_mini_cluster.cc:1428] Setting key 49087643b6919abe20d46d1daf7e72be
I20260620 23:03:20.701270 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:20.692991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:48215 (local address 127.17.188.126:45153)
0620 23:03:20.693066 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:20.693068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:20.693225 (+ 157us) server_negotiation.cc:407] Connection header received
0620 23:03:20.693708 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:20.693709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:20.693719 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:20.693769 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:20.694594 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.695066 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.695950 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.696126 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.696175 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:20.698048 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:20.698050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:20.698050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:20.698056 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:20.699802 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.700257 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.700259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.700259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.700318 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.700636 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.700638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.700638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.700789 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:20.700824 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:20.701090 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 23:03:20.701166 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:03:20.701457 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:20.692956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45153 (local address 127.17.188.66:48215)
0620 23:03:20.693116 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:20.693147 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:20.693625 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:20.693833 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:20.693836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:20.694343 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:20.694519 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:20.694523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.695137 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.695137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:20.695883 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:20.695884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.696189 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.696190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:20.696278 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:20.696686 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:20.696693 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:20.697969 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:20.699891 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:20.699897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:20.699899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:20.700183 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:20.700389 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:20.700390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:20.700390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:20.700523 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:20.700859 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:20.700863 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:20.700962 (+ 99us) client_negotiation.cc:769] Sending connection context
0620 23:03:20.701057 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 23:03:20.701195 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260620 23:03:20.701871 18538 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45153
I20260620 23:03:20.701964 18538 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:20.702134 18538 heartbeater.cc:511] Master 127.17.188.126:45153 requested a full tablet report, sending...
I20260620 23:03:20.702471 18213 ts_manager.cc:194] Registered new tserver with Master: bafe6cb940b04fdead8513e86b592f28 (127.17.188.66:46393)
I20260620 23:03:20.702996 18213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:48215
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:03:20.721578 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45153
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:20.800630 18543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:20.800874 18543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:20.800913 18543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:20.802016 18543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:20.802088 18543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:20.802134 18543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:20.803506 18543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45153
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:20.803756 18543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:20.803959 18543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:20.808705 18549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:20.808727 18548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:20.808743 18543 server_base.cc:1034] running on GCE node
W20260620 23:03:20.808727 18551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:20.809096 18543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:20.809367 18543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:20.810503 18543 hybrid_clock.cc:648] HybridClock initialized: now 1781996600810498 us; error 33 us; skew 500 ppm
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:20.812860 18543 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:03:20.813251 18543 webserver.cc:460] Webserver started at http://127.17.188.67:39067/ using document root <none> and password file <none>
I20260620 23:03:20.813496 18543 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:20.813565 18543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:20.813709 18543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:20.814792 18543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "5892c62cfcd343fb9749afc2c5c076e4"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "8896a961d4a7c7e1239eda254b3b324e"
server_key_iv: "9f122ff7a7a65c32999349c179a683be"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.815196 18543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "5892c62cfcd343fb9749afc2c5c076e4"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "8896a961d4a7c7e1239eda254b3b324e"
server_key_iv: "9f122ff7a7a65c32999349c179a683be"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.816531 18543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:20.817199 18558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.817420 18543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:20.817517 18543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "5892c62cfcd343fb9749afc2c5c076e4"
format_stamp: "Formatted at 2026-06-20 23:03:20 on dist-test-slave-zpss"
server_key: "8896a961d4a7c7e1239eda254b3b324e"
server_key_iv: "9f122ff7a7a65c32999349c179a683be"
server_key_version: "encryptionkey@0"
I20260620 23:03:20.817589 18543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:20.829504 18543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:20.831164 18543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:20.831308 18543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:20.831545 18543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:20.831828 18543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:20.831882 18543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.831920 18543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:20.831947 18543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:20.837407 18543 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:45885
I20260620 23:03:20.837469 18671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:45885 every 8 connection(s)
I20260620 23:03:20.837754 18543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:20.845456 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 18543
I20260620 23:03:20.845534 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:20.845729 18161 external_mini_cluster.cc:1428] Setting key a2bc834bfe8dedcb09b4f00f61111864
I20260620 23:03:20.847288 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:20.838418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:45607 (local address 127.17.188.126:45153)
0620 23:03:20.838460 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:20.838462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:20.838659 (+ 197us) server_negotiation.cc:407] Connection header received
0620 23:03:20.839237 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:20.839239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:20.839251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:20.839313 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:20.840186 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.840679 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.841716 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.841916 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.842008 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:20.844080 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:20.844081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:20.844082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:20.844089 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:20.845720 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.846226 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.846227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.846228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.846310 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.846624 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.846624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.846625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.846799 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:20.846854 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:20.847152 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 23:03:20.847229 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:03:20.847472 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:20.838391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45153 (local address 127.17.188.67:45607)
0620 23:03:20.838542 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:20.838580 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:20.839101 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:20.839376 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:20.839379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:20.839911 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:20.840082 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:20.840087 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.840773 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.840774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:20.841629 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:20.841631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.842030 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.842032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:20.842143 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:20.842645 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:20.842653 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:20.843988 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:20.845828 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:20.845833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:20.845835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:20.846114 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:20.846395 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:20.846396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:20.846396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:20.846556 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:20.846866 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:20.846871 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:20.847005 (+ 134us) client_negotiation.cc:769] Sending connection context
0620 23:03:20.847123 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 23:03:20.847274 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260620 23:03:20.847905 18672 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45153
I20260620 23:03:20.848022 18672 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:20.848192 18672 heartbeater.cc:511] Master 127.17.188.126:45153 requested a full tablet report, sending...
I20260620 23:03:20.848541 18213 ts_manager.cc:194] Registered new tserver with Master: 5892c62cfcd343fb9749afc2c5c076e4 (127.17.188.67:45885)
I20260620 23:03:20.848845 18213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:45607
I20260620 23:03:20.857764 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:20.880165 18679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:20.872693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59598 (local address 127.17.188.65:34967)
0620 23:03:20.872850 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:20.872855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:20.872866 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:03:20.872894 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:20.872896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:20.872917 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:20.873001 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:20.873611 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.874306 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.875037 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.875359 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.875441 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:20.877663 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:20.877669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:20.877669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:20.877683 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:20.878999 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.879367 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.879370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.879370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.879426 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.879613 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.879613 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.879614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.879736 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:20.879888 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:20.879964 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 23:03:20.880038 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
W20260620 23:03:20.880492 18318 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59598
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:20.900053 18679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:20.894286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.17.188.65:34967)
0620 23:03:20.894352 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:20.894354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:20.894362 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:20.894416 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:20.894416 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:20.894426 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:20.894472 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:20.894946 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.895401 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.896044 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:20.896222 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:20.896267 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:20.898199 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:20.898201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:20.898201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:20.898206 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:20.899233 (+ 1027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.899497 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.899498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.899498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.899539 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:20.899667 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:20.899667 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:20.899667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:20.899732 (+ 65us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:20.899803 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:20.899862 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 23:03:20.899951 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:03:20.900532 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 18275
I20260620 23:03:20.904981 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 18409
I20260620 23:03:20.909005 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 18543
I20260620 23:03:20.912982 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 18181
2026-06-20T23:03:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1828 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18686](info): set up 2 sockets
Jun 20 23:03:20 dist-test-slave-zpss krb5kdc[18686](info): commencing operation
krb5kdc: starting...
W20260620 23:03:21.920674 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:21 dist-test-slave-zpss krb5kdc[18686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:22.028046 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:36573
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:36573
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:22.106163 18699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:22.106351 18699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:22.106371 18699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:22.107486 18699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:22.107524 18699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:22.107538 18699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:22.107592 18699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:22.107614 18699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:22.109004 18699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:36573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:36573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:22.109237 18699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:22.109524 18699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:22.114243 18705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:22.114318 18704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:22.114339 18707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:22.114375 18699 server_base.cc:1034] running on GCE node
I20260620 23:03:22.114645 18699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:22.114964 18699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:22.116103 18699 hybrid_clock.cc:648] HybridClock initialized: now 1781996602116095 us; error 30 us; skew 500 ppm
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:22.118680 18699 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:22.119103 18699 webserver.cc:460] Webserver started at http://127.17.188.126:33031/ using document root <none> and password file <none>
I20260620 23:03:22.119304 18699 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:22.119369 18699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:22.119510 18699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:22.120448 18699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "6156c33846cd42b3be7c9dcd77082a3d"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "1eb47f30c992447855929ffef5499def"
server_key_iv: "5427adfdf1c06d70fc7306788f34f3dd"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.120796 18699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "6156c33846cd42b3be7c9dcd77082a3d"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "1eb47f30c992447855929ffef5499def"
server_key_iv: "5427adfdf1c06d70fc7306788f34f3dd"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.122159 18699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:22.122813 18714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.122959 18699 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:22.123052 18699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "6156c33846cd42b3be7c9dcd77082a3d"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "1eb47f30c992447855929ffef5499def"
server_key_iv: "5427adfdf1c06d70fc7306788f34f3dd"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.123127 18699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:22.137063 18699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:22.139330 18699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:22.139482 18699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:22.143388 18699 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:36573
I20260620 23:03:22.143415 18766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:36573 every 8 connection(s)
I20260620 23:03:22.143741 18699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:22.144516 18767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:22.146631 18767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d: Bootstrap starting.
I20260620 23:03:22.147114 18767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:22.147323 18767 log.cc:824] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:22.147909 18767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d: No bootstrap required, opened a new log
I20260620 23:03:22.149403 18767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6156c33846cd42b3be7c9dcd77082a3d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36573 } }
I20260620 23:03:22.149538 18767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:22.149560 18767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6156c33846cd42b3be7c9dcd77082a3d, State: Initialized, Role: FOLLOWER
I20260620 23:03:22.149653 18767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6156c33846cd42b3be7c9dcd77082a3d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36573 } }
I20260620 23:03:22.149729 18767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:22.149763 18767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:22.149816 18767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:22.150305 18767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6156c33846cd42b3be7c9dcd77082a3d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36573 } }
I20260620 23:03:22.150417 18767 leader_election.cc:302] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6156c33846cd42b3be7c9dcd77082a3d; no voters:
I20260620 23:03:22.150588 18767 leader_election.cc:288] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:22.150625 18770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:22.150787 18770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [term 1 LEADER]: Becoming Leader. State: Replica: 6156c33846cd42b3be7c9dcd77082a3d, State: Running, Role: LEADER
I20260620 23:03:22.150822 18767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:22.150879 18770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6156c33846cd42b3be7c9dcd77082a3d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36573 } }
I20260620 23:03:22.151355 18771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6156c33846cd42b3be7c9dcd77082a3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6156c33846cd42b3be7c9dcd77082a3d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36573 } } }
I20260620 23:03:22.151444 18771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:22.151608 18772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6156c33846cd42b3be7c9dcd77082a3d. Latest consensus state: current_term: 1 leader_uuid: "6156c33846cd42b3be7c9dcd77082a3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6156c33846cd42b3be7c9dcd77082a3d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36573 } } }
I20260620 23:03:22.151772 18772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:22.151891 18779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:22.152149 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 18699
I20260620 23:03:22.152251 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:22.152426 18779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:22.152443 18161 external_mini_cluster.cc:1428] Setting key 349e551ae3b86e527fb8b5d4df63b7c5
I20260620 23:03:22.154016 18779 catalog_manager.cc:1347] Generated new cluster ID: a537f9464ae04f41afa2c2065939b2fd
I20260620 23:03:22.154093 18779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:22.163911 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.153011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35166 (local address 127.17.188.126:36573)
0620 23:03:22.153476 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.153485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.153516 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:03:22.153635 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.153660 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.153727 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.153879 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:22.154636 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.155986 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.157092 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.157651 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.157735 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.160214 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:22.160218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:22.160219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:22.160230 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:22.162199 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.162561 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.162564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.162565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.162678 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.162873 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.162874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.162874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.163173 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:22.163347 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.163455 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.163694 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":65,"threads_started":1}
I20260620 23:03:22.164902 18779 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:22.165635 18779 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:22.175028 18779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6156c33846cd42b3be7c9dcd77082a3d: Generated new TSK 0
I20260620 23:03:22.175249 18779 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:03:22.199961 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36573
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:22.278154 18793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:22.278335 18793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:22.278357 18793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:22.279471 18793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:22.279512 18793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:22.279538 18793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:22.280972 18793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36573
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:22.281198 18793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:22.281445 18793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:22.286221 18801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:22.286237 18798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:22.286221 18799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:22.286509 18793 server_base.cc:1034] running on GCE node
I20260620 23:03:22.286731 18793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:22.286912 18793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:22.288072 18793 hybrid_clock.cc:648] HybridClock initialized: now 1781996602288061 us; error 25 us; skew 500 ppm
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:22.290591 18793 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:03:22.290988 18793 webserver.cc:460] Webserver started at http://127.17.188.65:46861/ using document root <none> and password file <none>
I20260620 23:03:22.291203 18793 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:22.291270 18793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:22.291415 18793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:22.292392 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "2b0708b653824951bab94e1a7374c3cd"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "8908a7b0ff4f41baf88be8eb56527f73"
server_key_iv: "18cbe58a177461e2fcb23753638c8c44"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.292734 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "2b0708b653824951bab94e1a7374c3cd"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "8908a7b0ff4f41baf88be8eb56527f73"
server_key_iv: "18cbe58a177461e2fcb23753638c8c44"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.294173 18793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:22.294838 18808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.294998 18793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:22.295091 18793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "2b0708b653824951bab94e1a7374c3cd"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "8908a7b0ff4f41baf88be8eb56527f73"
server_key_iv: "18cbe58a177461e2fcb23753638c8c44"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.295171 18793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:22.307838 18793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:22.309473 18793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:22.309620 18793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:22.309811 18793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:22.310092 18793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:22.310144 18793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.310175 18793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:22.310199 18793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.315701 18793 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:38271
I20260620 23:03:22.315735 18921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:38271 every 8 connection(s)
I20260620 23:03:22.316015 18793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:22.324117 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 18793
I20260620 23:03:22.324209 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:22.324347 18161 external_mini_cluster.cc:1428] Setting key a3228d9ad5656b90d2a1c2c17c785559
I20260620 23:03:22.324694 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.316787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:44157 (local address 127.17.188.126:36573)
0620 23:03:22.316856 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.316858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.317050 (+ 192us) server_negotiation.cc:407] Connection header received
0620 23:03:22.317943 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.317944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.317957 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.318008 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:22.318539 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.319012 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.319936 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.320130 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.320194 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.322255 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:22.322257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:22.322257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:22.322263 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:22.323405 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.323734 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.323735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.323736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.323782 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.324009 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.324009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.324010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.324147 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:22.324220 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.324524 (+ 304us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.324610 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 23:03:22.324848 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.316768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36573 (local address 127.17.188.65:44157)
0620 23:03:22.316934 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:22.316965 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:22.317858 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:22.318065 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:22.318069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:22.318298 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:22.318466 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.318469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.319111 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.319111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.319864 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.319865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.320212 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.320212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.320318 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.320846 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:22.320854 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:22.322176 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:22.323471 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:22.323476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:22.323477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:22.323674 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:22.323835 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:22.323836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:22.323836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:22.323920 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:22.324229 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:22.324235 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:22.324353 (+ 118us) client_negotiation.cc:769] Sending connection context
0620 23:03:22.324486 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 23:03:22.324638 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260620 23:03:22.325292 18922 heartbeater.cc:347] Connected to a master server at 127.17.188.126:36573
I20260620 23:03:22.325429 18922 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:22.325634 18922 heartbeater.cc:511] Master 127.17.188.126:36573 requested a full tablet report, sending...
I20260620 23:03:22.326139 18731 ts_manager.cc:194] Registered new tserver with Master: 2b0708b653824951bab94e1a7374c3cd (127.17.188.65:38271)
I20260620 23:03:22.326870 18731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:44157
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:03:22.345206 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36573
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:22.422951 18927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:22.423133 18927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:22.423154 18927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:22.424283 18927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:22.424322 18927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:22.424386 18927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:22.425858 18927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36573
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:22.426080 18927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:22.426295 18927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:22.430974 18932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:22.431032 18933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:22.431025 18935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:22.431182 18927 server_base.cc:1034] running on GCE node
I20260620 23:03:22.431429 18927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:22.431610 18927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:22.432760 18927 hybrid_clock.cc:648] HybridClock initialized: now 1781996602432744 us; error 31 us; skew 500 ppm
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:22.435144 18927 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:03:22.435539 18927 webserver.cc:460] Webserver started at http://127.17.188.66:35585/ using document root <none> and password file <none>
I20260620 23:03:22.435784 18927 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:22.435853 18927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:22.436002 18927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:22.436959 18927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "16ecc3b263c64c5393bc2dd1a5d0487b"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "8786072827c1f8e1e62460b3b6c5f809"
server_key_iv: "31719b7a76b8b021ef2c1d9956781948"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.437299 18927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "16ecc3b263c64c5393bc2dd1a5d0487b"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "8786072827c1f8e1e62460b3b6c5f809"
server_key_iv: "31719b7a76b8b021ef2c1d9956781948"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.438548 18927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:22.439260 18942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.439416 18927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:22.439510 18927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "16ecc3b263c64c5393bc2dd1a5d0487b"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "8786072827c1f8e1e62460b3b6c5f809"
server_key_iv: "31719b7a76b8b021ef2c1d9956781948"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.439589 18927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:22.457228 18927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:22.459444 18927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:22.459592 18927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:22.459805 18927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:22.460131 18927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:22.460187 18927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.460227 18927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:22.460255 18927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.466372 18927 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:42777
I20260620 23:03:22.466427 19055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:42777 every 8 connection(s)
I20260620 23:03:22.466813 18927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:22.469198 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 18927
I20260620 23:03:22.469282 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:22.469471 18161 external_mini_cluster.cc:1428] Setting key adac2d020debd2cbcc0e4a999cefd223
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:22.477258 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.467601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:47863 (local address 127.17.188.126:36573)
0620 23:03:22.467705 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.467707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.467856 (+ 149us) server_negotiation.cc:407] Connection header received
0620 23:03:22.468703 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.468704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.468713 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.468761 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:22.469528 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.470248 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.471788 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.472104 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.472182 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.474533 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:22.474534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:22.474534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:22.474540 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:22.475894 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.476359 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.476360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.476360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.476409 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.476683 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.476683 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.476684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.476777 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:22.476824 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.477140 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.477198 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 23:03:22.477557 19058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.467526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36573 (local address 127.17.188.66:47863)
0620 23:03:22.467743 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:22.467776 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:22.468621 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:22.468820 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:22.468823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:22.469139 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:22.469420 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.469427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.470330 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.470331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.471691 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.471694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.472188 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.472189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.472284 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.472750 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:22.472758 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:22.474448 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:22.475977 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:22.475985 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:22.475987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:22.476281 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:22.476472 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:22.476473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:22.476474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:22.476615 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:22.476851 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:22.476857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:22.477009 (+ 152us) client_negotiation.cc:769] Sending connection context
0620 23:03:22.477126 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 23:03:22.477278 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":145,"threads_started":1}
I20260620 23:03:22.478044 19056 heartbeater.cc:347] Connected to a master server at 127.17.188.126:36573
I20260620 23:03:22.478127 19056 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
I20260620 23:03:22.478302 19056 heartbeater.cc:511] Master 127.17.188.126:36573 requested a full tablet report, sending...
I20260620 23:03:22.478648 18731 ts_manager.cc:194] Registered new tserver with Master: 16ecc3b263c64c5393bc2dd1a5d0487b (127.17.188.66:42777)
I20260620 23:03:22.479022 18731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:47863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:03:22.493794 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36573
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:22.570128 19061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:22.570329 19061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:22.570364 19061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:22.571449 19061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:22.571508 19061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:22.571550 19061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:22.573031 19061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36573
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:22.573273 19061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:22.573509 19061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:22.578230 19066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:22.578249 19067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:22.578410 19061 server_base.cc:1034] running on GCE node
W20260620 23:03:22.578238 19069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:22.578660 19061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:22.578917 19061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:22.580068 19061 hybrid_clock.cc:648] HybridClock initialized: now 1781996602580060 us; error 35 us; skew 500 ppm
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:22.582438 19061 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:03:22.582826 19061 webserver.cc:460] Webserver started at http://127.17.188.67:42221/ using document root <none> and password file <none>
I20260620 23:03:22.583048 19061 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:22.583115 19061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:22.583249 19061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:22.584148 19061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "d10d6c18832442e990c166bdc8d057da"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "f6db0b7777ac6a0c449af2ebfe1078b0"
server_key_iv: "86f500d4816fa3a2df40e82d0eb3476d"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.584468 19061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "d10d6c18832442e990c166bdc8d057da"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "f6db0b7777ac6a0c449af2ebfe1078b0"
server_key_iv: "86f500d4816fa3a2df40e82d0eb3476d"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.585768 19061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:22.586412 19076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.586557 19061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:22.586649 19061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "d10d6c18832442e990c166bdc8d057da"
format_stamp: "Formatted at 2026-06-20 23:03:22 on dist-test-slave-zpss"
server_key: "f6db0b7777ac6a0c449af2ebfe1078b0"
server_key_iv: "86f500d4816fa3a2df40e82d0eb3476d"
server_key_version: "encryptionkey@0"
I20260620 23:03:22.586719 19061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:22.594431 19061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:22.596060 19061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:22.596208 19061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:22.596391 19061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:22.596647 19061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:22.596697 19061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.596735 19061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:22.596760 19061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:22.602201 19061 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:34749
I20260620 23:03:22.602233 19189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:34749 every 8 connection(s)
I20260620 23:03:22.602521 19061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:22.607808 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 19061
I20260620 23:03:22.607932 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:22.608101 18161 external_mini_cluster.cc:1428] Setting key dcf1215d5d8640266eb0d8c1d43a529a
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:22.610956 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.603193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:46601 (local address 127.17.188.126:36573)
0620 23:03:22.603260 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.603262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.603436 (+ 174us) server_negotiation.cc:407] Connection header received
0620 23:03:22.604244 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.604245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.604253 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.604301 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:22.604821 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.605294 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.606297 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.606474 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.606524 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.608668 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:22.608669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:22.608670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:22.608676 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:22.609908 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.610242 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.610243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.610243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.610288 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.610504 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.610505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.610505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.610597 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:22.610644 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.610845 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.610898 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:03:22.611099 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.603170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36573 (local address 127.17.188.67:46601)
0620 23:03:22.603332 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:22.603363 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:22.604167 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:22.604348 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:22.604351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:22.604591 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:22.604758 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.604761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.605460 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.605461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.606228 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.606230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.606540 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.606541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.606634 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.607149 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:22.607157 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:22.608573 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:22.609972 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:22.609977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:22.609978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:22.610181 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:22.610342 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:22.610343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:22.610343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:22.610438 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:22.610647 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:22.610650 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:22.610746 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 23:03:22.610818 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 23:03:22.610922 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260620 23:03:22.611586 19190 heartbeater.cc:347] Connected to a master server at 127.17.188.126:36573
I20260620 23:03:22.611675 19190 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:22.611812 19190 heartbeater.cc:511] Master 127.17.188.126:36573 requested a full tablet report, sending...
I20260620 23:03:22.612135 18731 ts_manager.cc:194] Registered new tserver with Master: d10d6c18832442e990c166bdc8d057da (127.17.188.67:34749)
I20260620 23:03:22.612493 18731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:46601
I20260620 23:03:22.619699 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:22.626199 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.621106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.17.188.126:36573)
0620 23:03:22.621153 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.621155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.621276 (+ 121us) server_negotiation.cc:407] Connection header received
0620 23:03:22.621369 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.621370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.621379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.621445 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:22.621961 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.622379 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.623009 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.623178 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.623224 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.624189 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:22.624190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:22.624190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:22.624196 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:22.625228 (+ 1032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.625539 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.625540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.625540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.625585 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.625823 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.625823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.625824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.625936 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:22.626004 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.626094 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.626146 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:03:22.627909 18731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:22.629405 18731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:22.635351 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.631283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:38271 (local address 127.0.0.1:52624)
0620 23:03:22.631378 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:22.631413 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:22.631484 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:22.631858 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:22.631861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:22.631863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:22.631925 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.631927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.633043 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.633044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.634190 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.634192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.635025 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.635026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.635113 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.635125 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:03:22.635197 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 23:03:22.635254 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":55,"threads_started":1}
I20260620 23:03:22.635409 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.631384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.17.188.65:38271)
0620 23:03:22.631656 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.631675 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.631685 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:03:22.631726 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.631728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.631765 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.631853 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:22.631994 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.632955 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.634283 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.634940 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.635026 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.635095 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.635272 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.635315 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":109,"threads_started":1}
I20260620 23:03:22.635679 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.631572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43482 (local address 127.17.188.67:34749)
0620 23:03:22.631976 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.631981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.631990 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:03:22.632025 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.632027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.632041 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.632134 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:22.632235 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.633461 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.634450 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.635099 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.635167 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.635214 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.635409 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.635596 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":106,"threads_started":1}
I20260620 23:03:22.635726 19202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.631614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:42777 (local address 127.0.0.1:39984)
0620 23:03:22.631825 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:22.631834 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:22.631876 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:22.632163 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:22.632164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:22.632165 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:22.632217 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.632219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.633533 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.633534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.634617 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.634619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.635531 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.635532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.635589 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.635593 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:22.635621 (+ 28us) client_negotiation.cc:240] Negotiation successful
0620 23:03:22.635656 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":32,"thread_start_us":83,"threads_started":1}
I20260620 23:03:22.635860 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.631490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:34749 (local address 127.0.0.1:43482)
0620 23:03:22.631572 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:22.631583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:22.631651 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:22.632139 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:22.632140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:22.632142 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:22.632176 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.632178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.633558 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.633559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.634367 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.634369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.635165 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.635166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.635221 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.635226 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:03:22.635270 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:03:22.635300 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":26,"thread_start_us":45,"threads_started":1}
I20260620 23:03:22.636036 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.631824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39984 (local address 127.17.188.66:42777)
0620 23:03:22.632000 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.632004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.632012 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:22.632047 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.632049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.632065 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.632291 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:22.632365 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.633427 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.634697 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.635433 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.635820 (+ 387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.635884 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.635910 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.635953 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260620 23:03:22.636103 18856 tablet_service.cc:1467] Processing CreateTablet for tablet 1c11cd00eaa44b419799754ebf20ea28 (DEFAULT_TABLE table=test-table [id=f22e65506ce841718e6c2ac557195343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:22.636451 18856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c11cd00eaa44b419799754ebf20ea28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:22.637904 19124 tablet_service.cc:1467] Processing CreateTablet for tablet 1c11cd00eaa44b419799754ebf20ea28 (DEFAULT_TABLE table=test-table [id=f22e65506ce841718e6c2ac557195343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:22.638036 18990 tablet_service.cc:1467] Processing CreateTablet for tablet 1c11cd00eaa44b419799754ebf20ea28 (DEFAULT_TABLE table=test-table [id=f22e65506ce841718e6c2ac557195343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:22.638172 19124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c11cd00eaa44b419799754ebf20ea28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:22.638336 18990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c11cd00eaa44b419799754ebf20ea28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:22.639698 19204 tablet_bootstrap.cc:492] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd: Bootstrap starting.
I20260620 23:03:22.640345 19204 tablet_bootstrap.cc:654] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:22.640820 19205 tablet_bootstrap.cc:492] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da: Bootstrap starting.
I20260620 23:03:22.640954 19204 log.cc:824] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:22.641562 19205 tablet_bootstrap.cc:654] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:22.641763 19207 tablet_bootstrap.cc:492] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b: Bootstrap starting.
I20260620 23:03:22.641857 19204 tablet_bootstrap.cc:492] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd: No bootstrap required, opened a new log
I20260620 23:03:22.641947 19204 ts_tablet_manager.cc:1397] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:03:22.642171 19205 log.cc:824] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:22.642453 19207 tablet_bootstrap.cc:654] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:22.643015 19207 log.cc:824] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:22.643258 19205 tablet_bootstrap.cc:492] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da: No bootstrap required, opened a new log
I20260620 23:03:22.643325 19205 ts_tablet_manager.cc:1397] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:03:22.643720 19207 tablet_bootstrap.cc:492] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b: No bootstrap required, opened a new log
I20260620 23:03:22.643787 19207 ts_tablet_manager.cc:1397] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:22.643888 19204 raft_consensus.cc:348] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } }
I20260620 23:03:22.644052 19204 raft_consensus.cc:374] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:22.644095 19204 raft_consensus.cc:729] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b0708b653824951bab94e1a7374c3cd, State: Initialized, Role: FOLLOWER
I20260620 23:03:22.644212 19204 consensus_queue.cc:260] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } }
I20260620 23:03:22.644475 19204 ts_tablet_manager.cc:1428] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:03:22.644594 18922 heartbeater.cc:503] Master 127.17.188.126:36573 was elected leader, sending a full tablet report...
I20260620 23:03:22.645222 19205 raft_consensus.cc:348] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } }
I20260620 23:03:22.645418 19205 raft_consensus.cc:374] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:22.645460 19205 raft_consensus.cc:729] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d10d6c18832442e990c166bdc8d057da, State: Initialized, Role: FOLLOWER
I20260620 23:03:22.645562 19205 consensus_queue.cc:260] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } }
I20260620 23:03:22.645824 19205 ts_tablet_manager.cc:1428] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:03:22.645843 19190 heartbeater.cc:503] Master 127.17.188.126:36573 was elected leader, sending a full tablet report...
I20260620 23:03:22.645747 19207 raft_consensus.cc:348] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } }
I20260620 23:03:22.645906 19207 raft_consensus.cc:374] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:22.645954 19207 raft_consensus.cc:729] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16ecc3b263c64c5393bc2dd1a5d0487b, State: Initialized, Role: FOLLOWER
I20260620 23:03:22.646060 19207 consensus_queue.cc:260] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } }
I20260620 23:03:22.646312 19056 heartbeater.cc:503] Master 127.17.188.126:36573 was elected leader, sending a full tablet report...
I20260620 23:03:22.646322 19207 ts_tablet_manager.cc:1428] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260620 23:03:22.717489 19057 tablet.cc:2369] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:03:22.816798 18923 tablet.cc:2369] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:03:22.853017 19191 tablet.cc:2369] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:22.860991 19212 raft_consensus.cc:482] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:22.861094 19212 raft_consensus.cc:504] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } }
I20260620 23:03:22.861405 19212 leader_election.cc:288] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b0708b653824951bab94e1a7374c3cd (127.17.188.65:38271), d10d6c18832442e990c166bdc8d057da (127.17.188.67:34749)
I20260620 23:03:22.865347 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.861789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:42075 (local address 127.17.188.67:34749)
0620 23:03:22.861828 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.861831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.861967 (+ 136us) server_negotiation.cc:407] Connection header received
0620 23:03:22.861993 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.861994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.862001 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.862066 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:22.862191 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.862888 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.864402 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.864960 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.865021 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.865027 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.865223 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.865258 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:22.865347 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.861699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:56883 (local address 127.17.188.65:38271)
0620 23:03:22.861762 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.861764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.861789 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:03:22.861816 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.861817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.861826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.861891 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:22.862061 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.862842 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.864402 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.864942 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.865018 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.865026 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.865223 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.865258 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:03:22.865375 19058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.861594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:38271 (local address 127.17.188.66:56883)
0620 23:03:22.861666 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:22.861685 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:22.861741 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:22.861912 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:22.861913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:22.861914 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:22.861987 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.861989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.862952 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.862953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.864311 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.864314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.865034 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.865035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.865116 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.865120 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:22.865196 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:03:22.865246 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":87}
I20260620 23:03:22.865382 19213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.861699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:34749 (local address 127.17.188.66:42075)
0620 23:03:22.861882 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:22.861891 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:22.861945 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:22.862066 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:22.862067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:22.862068 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:22.862128 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.862129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.862974 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.862975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.864311 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:22.864314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.865034 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.865035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:22.865116 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.865120 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:22.865206 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:03:22.865245 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":60,"thread_start_us":107,"threads_started":1}
I20260620 23:03:22.865671 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c11cd00eaa44b419799754ebf20ea28" candidate_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b0708b653824951bab94e1a7374c3cd" is_pre_election: true
I20260620 23:03:22.865795 19144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c11cd00eaa44b419799754ebf20ea28" candidate_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d10d6c18832442e990c166bdc8d057da" is_pre_election: true
I20260620 23:03:22.865839 18876 raft_consensus.cc:2454] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16ecc3b263c64c5393bc2dd1a5d0487b in term 0.
I20260620 23:03:22.865949 19144 raft_consensus.cc:2454] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16ecc3b263c64c5393bc2dd1a5d0487b in term 0.
I20260620 23:03:22.866106 18945 leader_election.cc:302] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16ecc3b263c64c5393bc2dd1a5d0487b, 2b0708b653824951bab94e1a7374c3cd; no voters:
I20260620 23:03:22.866252 19212 raft_consensus.cc:2781] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:22.866308 19212 raft_consensus.cc:482] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:22.866333 19212 raft_consensus.cc:3037] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:22.866966 19212 raft_consensus.cc:504] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } }
I20260620 23:03:22.867084 19212 leader_election.cc:288] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [CANDIDATE]: Term 1 election: Requested vote from peers 2b0708b653824951bab94e1a7374c3cd (127.17.188.65:38271), d10d6c18832442e990c166bdc8d057da (127.17.188.67:34749)
I20260620 23:03:22.867265 19144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c11cd00eaa44b419799754ebf20ea28" candidate_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d10d6c18832442e990c166bdc8d057da"
I20260620 23:03:22.867324 19144 raft_consensus.cc:3037] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:22.868064 19144 raft_consensus.cc:2454] T 1c11cd00eaa44b419799754ebf20ea28 P d10d6c18832442e990c166bdc8d057da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16ecc3b263c64c5393bc2dd1a5d0487b in term 1.
I20260620 23:03:22.868181 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c11cd00eaa44b419799754ebf20ea28" candidate_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b0708b653824951bab94e1a7374c3cd"
I20260620 23:03:22.868264 18945 leader_election.cc:302] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16ecc3b263c64c5393bc2dd1a5d0487b, d10d6c18832442e990c166bdc8d057da; no voters:
I20260620 23:03:22.868260 18876 raft_consensus.cc:3037] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:22.868348 19212 raft_consensus.cc:2781] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:22.868484 19212 raft_consensus.cc:686] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [term 1 LEADER]: Becoming Leader. State: Replica: 16ecc3b263c64c5393bc2dd1a5d0487b, State: Running, Role: LEADER
I20260620 23:03:22.868578 19212 consensus_queue.cc:237] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } }
I20260620 23:03:22.869088 18876 raft_consensus.cc:2454] T 1c11cd00eaa44b419799754ebf20ea28 P 2b0708b653824951bab94e1a7374c3cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16ecc3b263c64c5393bc2dd1a5d0487b in term 1.
I20260620 23:03:22.869290 18730 catalog_manager.cc:5495] T 1c11cd00eaa44b419799754ebf20ea28 P 16ecc3b263c64c5393bc2dd1a5d0487b reported cstate change: term changed from 0 to 1, leader changed from <none> to 16ecc3b263c64c5393bc2dd1a5d0487b (127.17.188.66). New cstate: current_term: 1 leader_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b0708b653824951bab94e1a7374c3cd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 38271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d10d6c18832442e990c166bdc8d057da" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16ecc3b263c64c5393bc2dd1a5d0487b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42777 } health_report { overall_health: HEALTHY } } }
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:22.920560 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.913230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.17.188.65:38271)
0620 23:03:22.913296 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.913299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.913425 (+ 126us) server_negotiation.cc:407] Connection header received
0620 23:03:22.913497 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.913498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.913509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.913583 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:22.914141 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.914640 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.915346 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.915535 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.915619 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.917827 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:22.917837 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:22.917837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:22.917843 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:22.919475 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.919775 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.919778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.919779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.919835 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.919975 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.919975 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.919975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.920092 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:22.920262 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.920312 (+ 50us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.920427 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:22.942220 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.935641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.17.188.65:38271)
0620 23:03:22.935701 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.935703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.935831 (+ 128us) server_negotiation.cc:407] Connection header received
0620 23:03:22.935878 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.935879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.935887 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.935935 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:22.936438 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.936894 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.937634 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.937811 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.937862 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.939909 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:22.939910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:22.939910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:22.939916 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:22.941270 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.941590 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.941590 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.941591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.941639 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.941799 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.941800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.941800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.941877 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:22.941940 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.942074 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.942130 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:03:22 dist-test-slave-zpss krb5kdc[18686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:22.962950 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:22.956746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.17.188.65:38271)
0620 23:03:22.956811 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:22.956813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:22.956848 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:03:22.956904 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:22.956905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:22.956913 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:22.956959 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:22.957504 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.957975 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.958622 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:22.958783 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:22.958828 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:22.960891 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:22.960891 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:22.960892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:22.960897 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:22.962113 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.962407 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.962408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.962409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.962453 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:22.962605 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:22.962605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:22.962606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:22.962685 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:22.962751 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:22.962841 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 23:03:22.962890 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260620 23:03:22.963450 18836 tablet_service.cc:2908] Rejecting scan request for tablet 1c11cd00eaa44b419799754ebf20ea28: Uninitialized: clean time has not yet been initialized
I20260620 23:03:22.964279 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 18793
I20260620 23:03:22.969049 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 18927
I20260620 23:03:22.973598 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 19061
I20260620 23:03:22.977813 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 18699
2026-06-20T23:03:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2065 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:23 dist-test-slave-zpss krb5kdc[19228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:23 dist-test-slave-zpss krb5kdc[19228](info): set up 2 sockets
Jun 20 23:03:23 dist-test-slave-zpss krb5kdc[19228](info): commencing operation
krb5kdc: starting...
W20260620 23:03:24.008736 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:24.117425 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:33935
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:33935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:24.195007 19241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:24.195190 19241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:24.195210 19241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:24.196364 19241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:24.196404 19241 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 23:03:24.196419 19241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:24.196431 19241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:24.196482 19241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:24.196499 19241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:24.197911 19241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:33935
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:33935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:24.198143 19241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:24.198361 19241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:24.202931 19247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:24.202942 19249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:24.203016 19241 server_base.cc:1034] running on GCE node
W20260620 23:03:24.202939 19246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:24.203300 19241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:24.203575 19241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:24.204731 19241 hybrid_clock.cc:648] HybridClock initialized: now 1781996604204714 us; error 44 us; skew 500 ppm
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:24.211942 19241 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:24.212410 19241 webserver.cc:460] Webserver started at http://127.17.188.126:42775/ using document root <none> and password file <none>
I20260620 23:03:24.212628 19241 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:24.212693 19241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:24.212841 19241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:24.213945 19241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "e3a44a2e5c10653b5b94d78cae8fbd38"
server_key_iv: "2b6b56aea79f6527909b32af5fc50eea"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.214304 19241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "e3a44a2e5c10653b5b94d78cae8fbd38"
server_key_iv: "2b6b56aea79f6527909b32af5fc50eea"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.215587 19241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:03:24.216290 19256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:24.216477 19241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:24.216565 19241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "e3a44a2e5c10653b5b94d78cae8fbd38"
server_key_iv: "2b6b56aea79f6527909b32af5fc50eea"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.216615 19241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:24.242599 19241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:24.244376 19241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:24.244477 19241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:24.248775 19241 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:33935
I20260620 23:03:24.248886 19318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:33935 every 8 connection(s)
I20260620 23:03:24.249174 19241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:24.249851 19319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:24.251622 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 19241
I20260620 23:03:24.251719 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:24.251886 18161 external_mini_cluster.cc:1428] Setting key c98e6004763a4f1171befda684a59712
I20260620 23:03:24.252434 19319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2: Bootstrap starting.
I20260620 23:03:24.253041 19319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:24.253394 19319 log.cc:824] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:24.254079 19319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2: No bootstrap required, opened a new log
I20260620 23:03:24.255837 19319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33935 } }
I20260620 23:03:24.256009 19319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:24.256099 19319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d69f3bde4f44be9a43ab06c8cb6b6b2, State: Initialized, Role: FOLLOWER
I20260620 23:03:24.256464 19319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33935 } }
I20260620 23:03:24.256568 19319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:24.256616 19319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:24.256666 19319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:24.257180 19319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33935 } }
I20260620 23:03:24.257304 19319 leader_election.cc:302] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d69f3bde4f44be9a43ab06c8cb6b6b2; no voters:
I20260620 23:03:24.257489 19319 leader_election.cc:288] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:24.257551 19324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:24.257714 19324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [term 1 LEADER]: Becoming Leader. State: Replica: 2d69f3bde4f44be9a43ab06c8cb6b6b2, State: Running, Role: LEADER
I20260620 23:03:24.257751 19319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:24.257802 19324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33935 } }
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:24.258152 19324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d69f3bde4f44be9a43ab06c8cb6b6b2. Latest consensus state: current_term: 1 leader_uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33935 } } }
I20260620 23:03:24.258231 19324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:24.258401 19325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69f3bde4f44be9a43ab06c8cb6b6b2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33935 } } }
I20260620 23:03:24.258539 19325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:24.258473 19329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:24.259053 19329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:24.262281 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.252386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.17.188.126:33935)
0620 23:03:24.252547 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.252556 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.252588 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:03:24.252991 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.252997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.253039 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.253183 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:24.253890 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.254787 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.255522 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.255902 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.256023 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.258552 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:24.258556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:24.258556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:24.258567 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:24.260679 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.261089 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.261092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.261092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.261199 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.261449 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.261450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.261450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.261730 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:24.261867 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.261943 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.262077 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260620 23:03:24.263013 19329 catalog_manager.cc:1347] Generated new cluster ID: 6a68cb37ec604180b9ccbad46aa01f65
I20260620 23:03:24.263067 19329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:24.281476 19329 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:24.282156 19329 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:24.291919 19329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2: Generated new TSK 0
I20260620 23:03:24.292104 19329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:03:24.322031 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:33935
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:24.418128 19345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:24.418326 19345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:24.418349 19345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:24.419601 19345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:24.419639 19345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:24.419660 19345 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:03:24.419677 19345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:24.421120 19345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.188.126:33935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:24.421284 19345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:24.421455 19345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:24.426119 19350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:24.426151 19345 server_base.cc:1034] running on GCE node
W20260620 23:03:24.426213 19351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:24.426461 19353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:24.426697 19345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:24.426903 19345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:24.428032 19345 hybrid_clock.cc:648] HybridClock initialized: now 1781996604428022 us; error 28 us; skew 500 ppm
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:24.430522 19345 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:03:24.430873 19345 webserver.cc:460] Webserver started at http://127.17.188.65:45953/ using document root <none> and password file <none>
I20260620 23:03:24.431064 19345 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:24.431103 19345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:24.431185 19345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:24.432097 19345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "6b4cbb2f014e4838b9e429d595abf331"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "f94e77848c59bed9137253b04369860c"
server_key_iv: "8c02434e12292112b906d203f81f5152"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.432375 19345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "6b4cbb2f014e4838b9e429d595abf331"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "f94e77848c59bed9137253b04369860c"
server_key_iv: "8c02434e12292112b906d203f81f5152"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.433600 19345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:24.434213 19360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:24.434422 19345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:24.434492 19345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "6b4cbb2f014e4838b9e429d595abf331"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "f94e77848c59bed9137253b04369860c"
server_key_iv: "8c02434e12292112b906d203f81f5152"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.434541 19345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:24.459784 19345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:24.461460 19345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:24.461560 19345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:24.461977 19345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:24.462011 19345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:24.462033 19345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:24.462087 19345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:24.468694 19345 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:34681
I20260620 23:03:24.468762 19475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:34681 every 8 connection(s)
I20260620 23:03:24.469053 19345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:24.471349 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.462463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:52545 (local address 127.17.188.126:33935)
0620 23:03:24.462533 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.462535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.462730 (+ 195us) server_negotiation.cc:407] Connection header received
0620 23:03:24.463177 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.463190 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.463203 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.463284 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:24.464040 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.464869 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.466023 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.466268 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.466348 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.468677 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:24.468678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:24.468679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:24.468687 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:24.470102 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.470511 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.470511 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.470512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.470559 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.470798 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.470799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.470800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.470933 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:24.470987 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.471223 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.471289 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:03:24.471493 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.462429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33935 (local address 127.17.188.65:52545)
0620 23:03:24.462595 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:24.462632 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:24.463078 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:24.463330 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:24.463333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:24.463688 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:24.463940 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.463946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.464994 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.464995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.465915 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.465918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.466367 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.466368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.466493 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.466981 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:24.466990 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:24.468569 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:24.470207 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.470211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.470213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.470448 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.470597 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.470597 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.470597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.470693 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.470994 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:24.470999 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:24.471113 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 23:03:24.471194 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 23:03:24.471298 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260620 23:03:24.471978 19476 heartbeater.cc:347] Connected to a master server at 127.17.188.126:33935
I20260620 23:03:24.472074 19476 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:24.472249 19476 heartbeater.cc:511] Master 127.17.188.126:33935 requested a full tablet report, sending...
I20260620 23:03:24.472724 19272 ts_manager.cc:194] Registered new tserver with Master: 6b4cbb2f014e4838b9e429d595abf331 (127.17.188.65:34681)
I20260620 23:03:24.473469 19272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:52545
I20260620 23:03:24.476939 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 19345
I20260620 23:03:24.477046 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:24.477205 18161 external_mini_cluster.cc:1428] Setting key d3645daea67394f33958799a6943ac26
I20260620 23:03:24.478049 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.473001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:60789 (local address 127.17.188.126:33935)
0620 23:03:24.473041 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.473043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.473138 (+ 95us) server_negotiation.cc:407] Connection header received
0620 23:03:24.473194 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.473195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.473203 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.473245 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:24.473580 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.474072 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.474738 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.474932 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.475014 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.475761 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:24.475762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:24.475762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:24.475769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:24.476864 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.477260 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.477262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.477262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.477316 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.477524 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.477525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.477525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.477645 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:24.477728 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.477861 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.477942 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:03:24.478049 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.472941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33935 (local address 127.17.188.65:60789)
0620 23:03:24.473057 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:24.473067 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:24.473138 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:24.473257 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:24.473258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:24.473451 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:24.473505 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.473507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.474162 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.474163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.474649 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.474651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.475025 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.475026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.475096 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.475416 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:24.475420 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:24.475675 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:24.476971 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.476972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.476972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.477177 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.477395 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.477396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.477396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.477454 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.477708 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:24.477709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:24.477770 (+ 61us) client_negotiation.cc:769] Sending connection context
0620 23:03:24.477822 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 23:03:24.477922 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:03:24.497602 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:33935
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:24.575343 19485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:24.575554 19485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:24.575577 19485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:24.576711 19485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:24.576752 19485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:24.576773 19485 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:03:24.576831 19485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:24.578202 19485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.188.126:33935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:24.578423 19485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:24.578625 19485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:24.583403 19491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:24.583506 19485 server_base.cc:1034] running on GCE node
W20260620 23:03:24.583408 19490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:24.583603 19493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:24.583791 19485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:24.584043 19485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:24.585188 19485 hybrid_clock.cc:648] HybridClock initialized: now 1781996604585180 us; error 26 us; skew 500 ppm
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:24.587538 19485 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:03:24.587949 19485 webserver.cc:460] Webserver started at http://127.17.188.66:36901/ using document root <none> and password file <none>
I20260620 23:03:24.588150 19485 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:24.588229 19485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:24.588376 19485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:24.589385 19485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "c1670ec8f6074934a5cd484291f409c2"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "010954b47be469f24a5fbb2c5d85c72a"
server_key_iv: "dc0df92d54cb032164fc51991ac3ff27"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.589737 19485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "c1670ec8f6074934a5cd484291f409c2"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "010954b47be469f24a5fbb2c5d85c72a"
server_key_iv: "dc0df92d54cb032164fc51991ac3ff27"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.590973 19485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:24.591714 19500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:24.591889 19485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:24.591997 19485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "c1670ec8f6074934a5cd484291f409c2"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "010954b47be469f24a5fbb2c5d85c72a"
server_key_iv: "dc0df92d54cb032164fc51991ac3ff27"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.592083 19485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:24.612164 19485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:24.614575 19485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:24.614729 19485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:24.615279 19485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:24.615336 19485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:24.615394 19485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:24.615433 19485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:24.622279 19485 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:41759
I20260620 23:03:24.622340 19615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:41759 every 8 connection(s)
I20260620 23:03:24.622645 19485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:24.624949 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.615713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:32803 (local address 127.17.188.126:33935)
0620 23:03:24.615758 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.615760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.615981 (+ 221us) server_negotiation.cc:407] Connection header received
0620 23:03:24.616433 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.616435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.616449 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.616529 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:24.617190 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.617800 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.618919 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.619222 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.619344 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.622542 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:24.622543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:24.622544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:24.622549 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:24.623897 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.624230 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.624231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.624231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.624295 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.624474 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.624474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.624475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.624565 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:24.624612 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.624826 (+ 214us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.624892 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:03:24.625104 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.615708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33935 (local address 127.17.188.66:32803)
0620 23:03:24.615848 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:24.615885 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:24.616328 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:24.616576 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:24.616579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:24.616864 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:24.617099 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.617104 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.617901 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.617902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.618844 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.618846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.619397 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.619398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.619555 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.620107 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:24.620124 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:24.622441 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:24.623964 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.623968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.623970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.624168 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.624334 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.624334 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.624335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.624427 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.624619 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:24.624622 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:24.624719 (+ 97us) client_negotiation.cc:769] Sending connection context
0620 23:03:24.624794 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 23:03:24.624899 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":83,"threads_started":1}
I20260620 23:03:24.625589 19616 heartbeater.cc:347] Connected to a master server at 127.17.188.126:33935
I20260620 23:03:24.625677 19616 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:24.625864 19616 heartbeater.cc:511] Master 127.17.188.126:33935 requested a full tablet report, sending...
I20260620 23:03:24.626255 19273 ts_manager.cc:194] Registered new tserver with Master: c1670ec8f6074934a5cd484291f409c2 (127.17.188.66:41759)
I20260620 23:03:24.626643 19273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:32803
I20260620 23:03:24.631604 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.626956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:55677 (local address 127.17.188.126:33935)
0620 23:03:24.626995 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.626997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.627114 (+ 117us) server_negotiation.cc:407] Connection header received
0620 23:03:24.627170 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.627171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.627178 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.627216 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:24.627555 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.627974 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.628567 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.628726 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.628770 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.629527 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:24.629528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:24.629528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:24.629533 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:24.630822 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.631068 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.631069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.631069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.631106 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.631215 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.631215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.631215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.631322 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:24.631365 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.631499 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.631550 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:03:24.631609 19622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.626872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33935 (local address 127.17.188.66:55677)
0620 23:03:24.627047 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:24.627058 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:24.627118 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:24.627231 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:24.627232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:24.627433 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:24.627491 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.627493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.628047 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.628048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.628507 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.628508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.628781 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.628782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.628833 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.629135 (+ 302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:24.629139 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:24.629460 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:24.630883 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.630883 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.630884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.631019 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.631142 (+ 123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.631142 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.631142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.631183 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.631370 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:24.631370 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:24.631435 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 23:03:24.631484 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:03:24.631549 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260620 23:03:24.631814 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 19485
I20260620 23:03:24.631877 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:24.632009 18161 external_mini_cluster.cc:1428] Setting key 2b237e9e51ce43d86075910677afed00
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:03:24.653967 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:33935
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:24.732226 19625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:24.732409 19625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:24.732431 19625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:24.733593 19625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:24.733631 19625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:24.733650 19625 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:03:24.733667 19625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:24.735150 19625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.188.126:33935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:24.735369 19625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:24.735618 19625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:24.740330 19633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:24.740324 19630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:24.740327 19631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:24.740473 19625 server_base.cc:1034] running on GCE node
I20260620 23:03:24.740797 19625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:24.741047 19625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:24.742245 19625 hybrid_clock.cc:648] HybridClock initialized: now 1781996604742237 us; error 24 us; skew 500 ppm
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:24.744529 19625 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:03:24.744937 19625 webserver.cc:460] Webserver started at http://127.17.188.67:46435/ using document root <none> and password file <none>
I20260620 23:03:24.745138 19625 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:24.745201 19625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:24.745389 19625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:24.746378 19625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "e78dc014564e46fc87a01073f3653f98"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "a299b9f08aa8be19b1f4111ed9f70dab"
server_key_iv: "b68a5a0cf68466672ebffe702f9bd6fb"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.746724 19625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "e78dc014564e46fc87a01073f3653f98"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "a299b9f08aa8be19b1f4111ed9f70dab"
server_key_iv: "b68a5a0cf68466672ebffe702f9bd6fb"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.747946 19625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:03:24.748606 19640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:24.748759 19625 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:24.748843 19625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "e78dc014564e46fc87a01073f3653f98"
format_stamp: "Formatted at 2026-06-20 23:03:24 on dist-test-slave-zpss"
server_key: "a299b9f08aa8be19b1f4111ed9f70dab"
server_key_iv: "b68a5a0cf68466672ebffe702f9bd6fb"
server_key_version: "encryptionkey@0"
I20260620 23:03:24.748920 19625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:24.763757 19625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:24.765424 19625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:24.765559 19625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:24.765997 19625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:24.766057 19625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:24.766095 19625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:24.766122 19625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:24.773104 19625 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:33991
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:24.773190 19755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:33991 every 8 connection(s)
I20260620 23:03:24.773509 19625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:24.776492 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.766469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:53143 (local address 127.17.188.126:33935)
0620 23:03:24.766510 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.766513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.766736 (+ 223us) server_negotiation.cc:407] Connection header received
0620 23:03:24.767231 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.767233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.767246 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.767304 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:24.768272 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.769054 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.770695 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.770955 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.771013 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.773721 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:24.773722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:24.773722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:24.773731 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:24.775000 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.775408 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.775410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.775410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.775471 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.775779 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.775780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.775781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.775968 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:24.776055 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.776317 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.776402 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:03:24.776682 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.766466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33935 (local address 127.17.188.67:53143)
0620 23:03:24.766626 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:24.766658 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:24.767121 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:24.767411 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:24.767416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:24.767866 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:24.768153 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.768159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.769157 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.769158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.770572 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.770576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.771019 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.771020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.771163 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.771758 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:24.771768 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:24.773615 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:24.775086 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.775092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.775094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.775313 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.775570 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.775570 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.775571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.775696 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.776043 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:24.776049 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:24.776185 (+ 136us) client_negotiation.cc:769] Sending connection context
0620 23:03:24.776293 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 23:03:24.776456 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260620 23:03:24.777216 19756 heartbeater.cc:347] Connected to a master server at 127.17.188.126:33935
I20260620 23:03:24.777293 19756 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:24.777508 19756 heartbeater.cc:511] Master 127.17.188.126:33935 requested a full tablet report, sending...
I20260620 23:03:24.777980 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 19625
I20260620 23:03:24.778077 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:24.778261 18161 external_mini_cluster.cc:1428] Setting key 88b393daa08294339bde3b34f3dd2781
I20260620 23:03:24.778469 19273 ts_manager.cc:194] Registered new tserver with Master: e78dc014564e46fc87a01073f3653f98 (127.17.188.67:33991)
I20260620 23:03:24.778980 19273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:53143
I20260620 23:03:24.779034 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:24.784209 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.778336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:46477 (local address 127.17.188.126:33935)
0620 23:03:24.778373 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.778374 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.778508 (+ 134us) server_negotiation.cc:407] Connection header received
0620 23:03:24.778676 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.778677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.778685 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.778712 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:24.779305 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.779752 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.780464 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.780759 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.780828 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.781710 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:24.781711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:24.781712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:24.781718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:24.783267 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.783568 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.783569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.783569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.783630 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.783768 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.783769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.783769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.783891 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:24.783954 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.784101 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.784155 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:03:24.784400 19762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.778269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33935 (local address 127.17.188.67:46477)
0620 23:03:24.778445 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:24.778455 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:24.778600 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:24.778765 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:24.778766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:24.779156 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:24.779231 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.779233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.779863 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.779864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.780383 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.780385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.780843 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.780844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.780911 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.781366 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:24.781370 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:24.781631 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:24.783341 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.783342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.783342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.783495 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.783688 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:24.783689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:24.783689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:24.783731 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:24.783957 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:24.783958 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:24.784006 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 23:03:24.784083 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 23:03:24.784187 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:03:24 dist-test-slave-zpss krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:24.800801 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.794930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55900 (local address 127.17.188.126:33935)
0620 23:03:24.794992 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.794994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.795033 (+ 39us) server_negotiation.cc:407] Connection header received
0620 23:03:24.795080 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.795080 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.795087 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.795132 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:24.795644 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.796068 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.796741 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.796913 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.796959 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.798971 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:24.798972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:24.798973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:24.798978 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:24.800050 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.800320 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.800321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.800321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.800365 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:24.800498 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:24.800498 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:24.800499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:24.800571 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:24.800630 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.800705 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.800750 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:03:24.802239 19273 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:24.803683 19273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:24.809579 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.805728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:34681 (local address 127.0.0.1:43436)
0620 23:03:24.805849 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:24.805899 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:24.805963 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:24.806276 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:24.806279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:24.806282 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:24.806338 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.806340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.807338 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.807339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.808301 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.808303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.809218 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.809219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.809292 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.809304 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:03:24.809450 (+ 146us) client_negotiation.cc:240] Negotiation successful
0620 23:03:24.809502 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":73,"threads_started":1}
I20260620 23:03:24.809594 19776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.805924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43436 (local address 127.17.188.65:34681)
0620 23:03:24.806097 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.806101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.806111 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:03:24.806150 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.806152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.806185 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.806272 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:24.806446 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.807257 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.808405 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.809125 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.809207 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.809263 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.809467 (+ 204us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.809503 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
I20260620 23:03:24.810164 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.805948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33991 (local address 127.0.0.1:40256)
0620 23:03:24.806273 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:24.806282 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:24.806327 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:24.806539 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:24.806540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:24.806542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:24.806589 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.806590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.807817 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.807818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.808746 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.808748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.809897 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.809898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.809992 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.810008 (+ 16us) client_negotiation.cc:769] Sending connection context
0620 23:03:24.810050 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 23:03:24.810090 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":29,"thread_start_us":59,"threads_started":1}
I20260620 23:03:24.810284 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.805879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56628 (local address 127.17.188.66:41759)
0620 23:03:24.806406 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.806410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.806421 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:03:24.806456 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.806458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.806473 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.806645 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:24.806815 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.807849 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.809080 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.809842 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.809900 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.809968 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.810148 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.810188 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":73,"threads_started":1}
I20260620 23:03:24.810250 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.805770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:41759 (local address 127.0.0.1:56628)
0620 23:03:24.806110 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:24.806121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:24.806178 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:24.806701 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:24.806702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:24.806704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:24.806739 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.806740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.807929 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.807930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.808991 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:24.808994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.810010 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.810012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:24.810067 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.810070 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:24.810122 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 23:03:24.810151 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":78,"threads_started":1}
I20260620 23:03:24.810257 19410 tablet_service.cc:1467] Processing CreateTablet for tablet 994badc3e6ce4c749c73cd34f0a01f4c (DEFAULT_TABLE table=test-table [id=2df8f522d1d64ff58f63280005ee46e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:24.810645 19410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 994badc3e6ce4c749c73cd34f0a01f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:24.810725 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:24.806099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.17.188.67:33991)
0620 23:03:24.806272 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:24.806277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:24.806319 (+ 42us) server_negotiation.cc:407] Connection header received
0620 23:03:24.806384 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:24.806386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:24.806400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:24.806485 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:24.806661 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.807726 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.808858 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:24.809562 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:24.810447 (+ 885us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:24.810514 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:24.810538 (+ 24us) server_negotiation.cc:299] Negotiation successful
0620 23:03:24.810632 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260620 23:03:24.811110 19550 tablet_service.cc:1467] Processing CreateTablet for tablet 994badc3e6ce4c749c73cd34f0a01f4c (DEFAULT_TABLE table=test-table [id=2df8f522d1d64ff58f63280005ee46e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:24.811228 19675 tablet_service.cc:1467] Processing CreateTablet for tablet 994badc3e6ce4c749c73cd34f0a01f4c (DEFAULT_TABLE table=test-table [id=2df8f522d1d64ff58f63280005ee46e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:24.811421 19550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 994badc3e6ce4c749c73cd34f0a01f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:24.811574 19675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 994badc3e6ce4c749c73cd34f0a01f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:24.814091 19779 tablet_bootstrap.cc:492] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: Bootstrap starting.
I20260620 23:03:24.814910 19781 tablet_bootstrap.cc:492] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: Bootstrap starting.
I20260620 23:03:24.815001 19780 tablet_bootstrap.cc:492] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: Bootstrap starting.
I20260620 23:03:24.814970 19779 tablet_bootstrap.cc:654] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:24.815333 19779 log.cc:824] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:24.815521 19781 tablet_bootstrap.cc:654] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:24.815537 19780 tablet_bootstrap.cc:654] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:24.815840 19780 log.cc:824] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:24.815840 19781 log.cc:824] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:24.816185 19779 tablet_bootstrap.cc:492] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: No bootstrap required, opened a new log
I20260620 23:03:24.816264 19779 ts_tablet_manager.cc:1397] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:24.816987 19781 tablet_bootstrap.cc:492] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: No bootstrap required, opened a new log
I20260620 23:03:24.817058 19781 ts_tablet_manager.cc:1397] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:24.817363 19780 tablet_bootstrap.cc:492] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: No bootstrap required, opened a new log
I20260620 23:03:24.817473 19780 ts_tablet_manager.cc:1397] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:03:24.817916 19779 raft_consensus.cc:348] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } }
I20260620 23:03:24.818061 19779 raft_consensus.cc:374] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:24.818102 19779 raft_consensus.cc:729] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78dc014564e46fc87a01073f3653f98, State: Initialized, Role: FOLLOWER
I20260620 23:03:24.818217 19779 consensus_queue.cc:260] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } }
I20260620 23:03:24.818499 19756 heartbeater.cc:503] Master 127.17.188.126:33935 was elected leader, sending a full tablet report...
I20260620 23:03:24.818503 19779 ts_tablet_manager.cc:1428] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:03:24.819016 19781 raft_consensus.cc:348] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } }
I20260620 23:03:24.819167 19781 raft_consensus.cc:374] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:24.819211 19781 raft_consensus.cc:729] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1670ec8f6074934a5cd484291f409c2, State: Initialized, Role: FOLLOWER
I20260620 23:03:24.819320 19781 consensus_queue.cc:260] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } }
I20260620 23:03:24.819591 19781 ts_tablet_manager.cc:1428] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 23:03:24.819643 19616 heartbeater.cc:503] Master 127.17.188.126:33935 was elected leader, sending a full tablet report...
I20260620 23:03:24.819460 19780 raft_consensus.cc:348] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } }
I20260620 23:03:24.819881 19780 raft_consensus.cc:374] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:24.819911 19780 raft_consensus.cc:729] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4cbb2f014e4838b9e429d595abf331, State: Initialized, Role: FOLLOWER
I20260620 23:03:24.820024 19780 consensus_queue.cc:260] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } }
I20260620 23:03:24.820274 19780 ts_tablet_manager.cc:1428] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:03:24.820407 19476 heartbeater.cc:503] Master 127.17.188.126:33935 was elected leader, sending a full tablet report...
W20260620 23:03:24.873358 19617 tablet.cc:2369] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:03:24.969800 19477 tablet.cc:2369] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:03:25.024080 19757 tablet.cc:2369] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:25.115238 19787 raft_consensus.cc:482] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:25.115418 19787 raft_consensus.cc:504] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } }
I20260620 23:03:25.115818 19787 leader_election.cc:288] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e78dc014564e46fc87a01073f3653f98 (127.17.188.67:33991), c1670ec8f6074934a5cd484291f409c2 (127.17.188.66:41759)
I20260620 23:03:25.119735 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.116116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:44223 (local address 127.17.188.66:41759)
0620 23:03:25.116165 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.116167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.116226 (+ 59us) server_negotiation.cc:407] Connection header received
0620 23:03:25.116311 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.116312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.116320 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.116391 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:25.116581 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.117350 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.118831 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.119366 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.119445 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.119455 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.119630 (+ 175us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.119667 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:03:25.119735 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.116127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:52867 (local address 127.17.188.67:33991)
0620 23:03:25.116172 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.116174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.116226 (+ 52us) server_negotiation.cc:407] Connection header received
0620 23:03:25.116299 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.116300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.116309 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.116391 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:25.116581 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.117353 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.118830 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.119403 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.119467 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.119474 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.119630 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.119667 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:03:25.119774 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.116013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33991 (local address 127.17.188.65:52867)
0620 23:03:25.116170 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.116179 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.116236 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.116409 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.116409 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.116413 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:25.116496 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.116499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.117432 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.117433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.118749 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.118752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.119466 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.119466 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.119548 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.119554 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.119614 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.119665 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":79,"spinlock_wait_cycles":23424,"thread_start_us":85,"threads_started":1}
I20260620 23:03:25.119776 19788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.116011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:41759 (local address 127.17.188.65:44223)
0620 23:03:25.116148 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.116162 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.116236 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.116409 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.116409 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.116412 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:25.116497 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.116499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.117433 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.117433 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.118760 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.118762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.119465 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.119466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.119548 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.119555 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.119614 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.119665 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":67,"thread_start_us":74,"threads_started":1}
I20260620 23:03:25.120096 19710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994badc3e6ce4c749c73cd34f0a01f4c" candidate_uuid: "6b4cbb2f014e4838b9e429d595abf331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e78dc014564e46fc87a01073f3653f98" is_pre_election: true
I20260620 23:03:25.120275 19710 raft_consensus.cc:2454] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4cbb2f014e4838b9e429d595abf331 in term 0.
I20260620 23:03:25.120263 19570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994badc3e6ce4c749c73cd34f0a01f4c" candidate_uuid: "6b4cbb2f014e4838b9e429d595abf331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1670ec8f6074934a5cd484291f409c2" is_pre_election: true
I20260620 23:03:25.120380 19570 raft_consensus.cc:2454] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4cbb2f014e4838b9e429d595abf331 in term 0.
I20260620 23:03:25.120616 19364 leader_election.cc:302] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4cbb2f014e4838b9e429d595abf331, c1670ec8f6074934a5cd484291f409c2; no voters:
I20260620 23:03:25.120743 19787 raft_consensus.cc:2781] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:25.120829 19787 raft_consensus.cc:482] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:25.120865 19787 raft_consensus.cc:3037] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:25.121628 19787 raft_consensus.cc:504] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } }
I20260620 23:03:25.121767 19787 leader_election.cc:288] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [CANDIDATE]: Term 1 election: Requested vote from peers e78dc014564e46fc87a01073f3653f98 (127.17.188.67:33991), c1670ec8f6074934a5cd484291f409c2 (127.17.188.66:41759)
I20260620 23:03:25.121901 19710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994badc3e6ce4c749c73cd34f0a01f4c" candidate_uuid: "6b4cbb2f014e4838b9e429d595abf331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e78dc014564e46fc87a01073f3653f98"
I20260620 23:03:25.121904 19570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994badc3e6ce4c749c73cd34f0a01f4c" candidate_uuid: "6b4cbb2f014e4838b9e429d595abf331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1670ec8f6074934a5cd484291f409c2"
I20260620 23:03:25.121973 19570 raft_consensus.cc:3037] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:25.121994 19710 raft_consensus.cc:3037] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:25.122617 19570 raft_consensus.cc:2454] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4cbb2f014e4838b9e429d595abf331 in term 1.
I20260620 23:03:25.122671 19710 raft_consensus.cc:2454] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4cbb2f014e4838b9e429d595abf331 in term 1.
I20260620 23:03:25.122790 19364 leader_election.cc:302] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4cbb2f014e4838b9e429d595abf331, c1670ec8f6074934a5cd484291f409c2; no voters:
I20260620 23:03:25.122917 19787 raft_consensus.cc:2781] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:25.123076 19787 raft_consensus.cc:686] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 1 LEADER]: Becoming Leader. State: Replica: 6b4cbb2f014e4838b9e429d595abf331, State: Running, Role: LEADER
I20260620 23:03:25.123183 19787 consensus_queue.cc:237] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } }
I20260620 23:03:25.123888 19273 catalog_manager.cc:5495] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b4cbb2f014e4838b9e429d595abf331 (127.17.188.65). New cstate: current_term: 1 leader_uuid: "6b4cbb2f014e4838b9e429d595abf331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:25.161515 19776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.158906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43450 (local address 127.17.188.65:34681)
0620 23:03:25.158943 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.158945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.159000 (+ 55us) server_negotiation.cc:407] Connection header received
0620 23:03:25.159068 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.159069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.159083 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.159180 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:25.159338 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.159815 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.160622 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.160836 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.160905 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.161134 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:25.161225 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:25.161316 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.161391 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.161419 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:03:25.163115 19710 raft_consensus.cc:1261] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [term 1 FOLLOWER]: Refusing update from remote peer 6b4cbb2f014e4838b9e429d595abf331: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:25.163115 19570 raft_consensus.cc:1261] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Refusing update from remote peer 6b4cbb2f014e4838b9e429d595abf331: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:25.163399 19787 consensus_queue.cc:1035] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:25.163549 19790 consensus_queue.cc:1035] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:25.166584 19794 mvcc.cc:205] Tried to move back new op lower bound from 7299058094745464832 to 7299058094585024512. Current Snapshot: MvccSnapshot[applied={T|T < 7299058094745464832}]
I20260620 23:03:25.166692 19795 mvcc.cc:205] Tried to move back new op lower bound from 7299058094745464832 to 7299058094585024512. Current Snapshot: MvccSnapshot[applied={T|T < 7299058094745464832 or (T in {7299058094745464832})}]
I20260620 23:03:25.168856 19272 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:25.168932 19272 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:25.169706 19272 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2: Sending DeleteTablet for 3 replicas of tablet 994badc3e6ce4c749c73cd34f0a01f4c
I20260620 23:03:25.169997 19410 tablet_service.cc:1514] Processing DeleteTablet for tablet 994badc3e6ce4c749c73cd34f0a01f4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:25 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43436
I20260620 23:03:25.170006 19550 tablet_service.cc:1514] Processing DeleteTablet for tablet 994badc3e6ce4c749c73cd34f0a01f4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:25 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56628
I20260620 23:03:25.170065 19675 tablet_service.cc:1514] Processing DeleteTablet for tablet 994badc3e6ce4c749c73cd34f0a01f4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:25 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:40256
I20260620 23:03:25.170282 19802 tablet_replica.cc:330] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: stopping tablet replica
I20260620 23:03:25.170271 19272 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:25.170395 19272 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:25.170409 19802 raft_consensus.cc:2229] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:25.170516 19802 raft_consensus.cc:2258] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:25.171033 19802 ts_tablet_manager.cc:1905] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:25.171523 19410 tablet_service.cc:1467] Processing CreateTablet for tablet 652e87d9edaa4cf0b9460a8e7654e500 (DEFAULT_TABLE table=test-table [id=525c8da595e64a2db57b71141ba85282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:25.171562 19550 tablet_service.cc:1467] Processing CreateTablet for tablet 652e87d9edaa4cf0b9460a8e7654e500 (DEFAULT_TABLE table=test-table [id=525c8da595e64a2db57b71141ba85282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:25.171651 19410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 652e87d9edaa4cf0b9460a8e7654e500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:25.171664 19550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 652e87d9edaa4cf0b9460a8e7654e500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:25.171887 19675 tablet_service.cc:1467] Processing CreateTablet for tablet 652e87d9edaa4cf0b9460a8e7654e500 (DEFAULT_TABLE table=test-table [id=525c8da595e64a2db57b71141ba85282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:25.172003 19675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 652e87d9edaa4cf0b9460a8e7654e500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:25.173018 19780 tablet_bootstrap.cc:492] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: Bootstrap starting.
I20260620 23:03:25.173050 19802 ts_tablet_manager.cc:1918] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:25.173112 19781 tablet_bootstrap.cc:492] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: Bootstrap starting.
I20260620 23:03:25.173252 19802 log.cc:1196] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/994badc3e6ce4c749c73cd34f0a01f4c
I20260620 23:03:25.173445 19780 tablet_bootstrap.cc:654] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:25.173580 19781 tablet_bootstrap.cc:654] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:25.173833 19802 ts_tablet_manager.cc:1939] T 994badc3e6ce4c749c73cd34f0a01f4c P c1670ec8f6074934a5cd484291f409c2: Deleting consensus metadata
I20260620 23:03:25.173867 19779 tablet_bootstrap.cc:492] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: Bootstrap starting.
I20260620 23:03:25.174067 19780 tablet_bootstrap.cc:492] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: No bootstrap required, opened a new log
I20260620 23:03:25.174117 19780 ts_tablet_manager.cc:1397] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:25.174279 19779 tablet_bootstrap.cc:654] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:25.174274 19780 raft_consensus.cc:348] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } }
I20260620 23:03:25.174329 19780 raft_consensus.cc:374] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:25.174356 19780 raft_consensus.cc:729] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4cbb2f014e4838b9e429d595abf331, State: Initialized, Role: FOLLOWER
I20260620 23:03:25.174394 19780 consensus_queue.cc:260] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } }
I20260620 23:03:25.174499 19780 ts_tablet_manager.cc:1428] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:25.174858 19260 catalog_manager.cc:4841] TS c1670ec8f6074934a5cd484291f409c2 (127.17.188.66:41759): tablet 994badc3e6ce4c749c73cd34f0a01f4c (table test-table [id=2df8f522d1d64ff58f63280005ee46e4]) successfully deleted
I20260620 23:03:25.174896 19779 tablet_bootstrap.cc:492] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: No bootstrap required, opened a new log
I20260620 23:03:25.174937 19779 ts_tablet_manager.cc:1397] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:25.175086 19779 raft_consensus.cc:348] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } }
I20260620 23:03:25.175163 19779 raft_consensus.cc:374] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:25.175184 19779 raft_consensus.cc:729] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78dc014564e46fc87a01073f3653f98, State: Initialized, Role: FOLLOWER
I20260620 23:03:25.175221 19779 consensus_queue.cc:260] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } }
I20260620 23:03:25.175299 19779 ts_tablet_manager.cc:1428] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:25.175401 19781 tablet_bootstrap.cc:492] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: No bootstrap required, opened a new log
I20260620 23:03:25.175482 19781 ts_tablet_manager.cc:1397] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:03:25.175755 19781 raft_consensus.cc:348] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } }
I20260620 23:03:25.175829 19781 raft_consensus.cc:374] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:25.175855 19781 raft_consensus.cc:729] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1670ec8f6074934a5cd484291f409c2, State: Initialized, Role: FOLLOWER
I20260620 23:03:25.175915 19781 consensus_queue.cc:260] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } }
I20260620 23:03:25.176014 19781 ts_tablet_manager.cc:1428] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:25.177433 19803 tablet_replica.cc:330] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: stopping tablet replica
I20260620 23:03:25.177527 19803 raft_consensus.cc:2229] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:25.177623 19803 raft_consensus.cc:2258] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:25.177840 19804 tablet_replica.cc:330] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: stopping tablet replica
I20260620 23:03:25.177934 19804 raft_consensus.cc:2229] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:25.178030 19804 raft_consensus.cc:2258] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:25.178087 19803 ts_tablet_manager.cc:1905] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:25.178450 19804 ts_tablet_manager.cc:1905] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:25.179293 19803 ts_tablet_manager.cc:1918] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:25.179378 19803 log.cc:1196] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/994badc3e6ce4c749c73cd34f0a01f4c
I20260620 23:03:25.179605 19803 ts_tablet_manager.cc:1939] T 994badc3e6ce4c749c73cd34f0a01f4c P 6b4cbb2f014e4838b9e429d595abf331: Deleting consensus metadata
I20260620 23:03:25.179972 19258 catalog_manager.cc:4841] TS 6b4cbb2f014e4838b9e429d595abf331 (127.17.188.65:34681): tablet 994badc3e6ce4c749c73cd34f0a01f4c (table test-table [id=2df8f522d1d64ff58f63280005ee46e4]) successfully deleted
I20260620 23:03:25.180215 19804 ts_tablet_manager.cc:1918] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:25.180284 19804 log.cc:1196] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/994badc3e6ce4c749c73cd34f0a01f4c
I20260620 23:03:25.180541 19804 ts_tablet_manager.cc:1939] T 994badc3e6ce4c749c73cd34f0a01f4c P e78dc014564e46fc87a01073f3653f98: Deleting consensus metadata
I20260620 23:03:25.180914 19258 catalog_manager.cc:4841] TS e78dc014564e46fc87a01073f3653f98 (127.17.188.67:33991): tablet 994badc3e6ce4c749c73cd34f0a01f4c (table test-table [id=2df8f522d1d64ff58f63280005ee46e4]) successfully deleted
I20260620 23:03:25.210830 19785 raft_consensus.cc:482] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:25.210934 19785 raft_consensus.cc:504] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } }
I20260620 23:03:25.211283 19785 leader_election.cc:288] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b4cbb2f014e4838b9e429d595abf331 (127.17.188.65:34681), c1670ec8f6074934a5cd484291f409c2 (127.17.188.66:41759)
I20260620 23:03:25.214577 19808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.211429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:41759 (local address 127.17.188.67:48451)
0620 23:03:25.211551 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.211559 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.211597 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.211831 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.211832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.211834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:25.211932 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.211934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.212765 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.212766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.213614 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.213616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.214305 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.214306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.214377 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.214382 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.214446 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.214484 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":10,"thread_start_us":61,"threads_started":1}
I20260620 23:03:25.214579 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.211676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:48451 (local address 127.17.188.66:41759)
0620 23:03:25.211715 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.211717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.211724 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:03:25.211744 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.211745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.211754 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.211809 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:25.212027 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.212702 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.213774 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.214214 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.214285 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.214294 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.214461 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.214496 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:03:25.214921 19776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.211479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:44825 (local address 127.17.188.65:34681)
0620 23:03:25.211519 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.211521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.211529 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:25.211625 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.211625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.211635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.211693 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:25.211881 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.212637 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.214019 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.214506 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.214569 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.214575 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.214831 (+ 256us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.214863 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:25.214938 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.211421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:34681 (local address 127.17.188.67:44825)
0620 23:03:25.211482 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.211494 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.211537 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.211708 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.211709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.211710 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:25.211783 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.211785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.212722 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.212723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.213933 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.213935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.214606 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.214607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.214701 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.214704 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.214822 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.214862 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 23:03:25.215217 19570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652e87d9edaa4cf0b9460a8e7654e500" candidate_uuid: "e78dc014564e46fc87a01073f3653f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1670ec8f6074934a5cd484291f409c2" is_pre_election: true
I20260620 23:03:25.215332 19570 raft_consensus.cc:2454] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e78dc014564e46fc87a01073f3653f98 in term 0.
I20260620 23:03:25.215284 19430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652e87d9edaa4cf0b9460a8e7654e500" candidate_uuid: "e78dc014564e46fc87a01073f3653f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4cbb2f014e4838b9e429d595abf331" is_pre_election: true
I20260620 23:03:25.215401 19430 raft_consensus.cc:2454] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e78dc014564e46fc87a01073f3653f98 in term 0.
I20260620 23:03:25.215512 19644 leader_election.cc:302] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1670ec8f6074934a5cd484291f409c2, e78dc014564e46fc87a01073f3653f98; no voters:
I20260620 23:03:25.215672 19785 raft_consensus.cc:2781] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:25.215732 19785 raft_consensus.cc:482] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:25.215765 19785 raft_consensus.cc:3037] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:25.216399 19785 raft_consensus.cc:504] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } }
I20260620 23:03:25.216554 19785 leader_election.cc:288] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [CANDIDATE]: Term 1 election: Requested vote from peers 6b4cbb2f014e4838b9e429d595abf331 (127.17.188.65:34681), c1670ec8f6074934a5cd484291f409c2 (127.17.188.66:41759)
I20260620 23:03:25.216717 19570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652e87d9edaa4cf0b9460a8e7654e500" candidate_uuid: "e78dc014564e46fc87a01073f3653f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1670ec8f6074934a5cd484291f409c2"
I20260620 23:03:25.216746 19430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652e87d9edaa4cf0b9460a8e7654e500" candidate_uuid: "e78dc014564e46fc87a01073f3653f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4cbb2f014e4838b9e429d595abf331"
I20260620 23:03:25.216812 19430 raft_consensus.cc:3037] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:25.216811 19570 raft_consensus.cc:3037] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:25.217454 19570 raft_consensus.cc:2454] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e78dc014564e46fc87a01073f3653f98 in term 1.
I20260620 23:03:25.217610 19644 leader_election.cc:302] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1670ec8f6074934a5cd484291f409c2, e78dc014564e46fc87a01073f3653f98; no voters:
I20260620 23:03:25.217707 19785 raft_consensus.cc:2781] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:25.217823 19785 raft_consensus.cc:686] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 1 LEADER]: Becoming Leader. State: Replica: e78dc014564e46fc87a01073f3653f98, State: Running, Role: LEADER
I20260620 23:03:25.217891 19785 consensus_queue.cc:237] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } }
I20260620 23:03:25.217926 19430 raft_consensus.cc:2454] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e78dc014564e46fc87a01073f3653f98 in term 1.
I20260620 23:03:25.218551 19272 catalog_manager.cc:5495] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 reported cstate change: term changed from 0 to 1, leader changed from <none> to e78dc014564e46fc87a01073f3653f98 (127.17.188.67). New cstate: current_term: 1 leader_uuid: "e78dc014564e46fc87a01073f3653f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } health_report { overall_health: HEALTHY } } }
Jun 20 23:03:25 dist-test-slave-zpss krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:25.245307 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.237986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33935 (local address 127.0.0.1:55908)
0620 23:03:25.238176 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.238184 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.238252 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.238401 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.238402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.238683 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:25.238726 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.238728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.239644 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.239644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.240185 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.240186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.240593 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.240594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.240628 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.241190 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:25.241194 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:25.242678 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:25.244375 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:25.244381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:25.244382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:25.244557 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:25.244767 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:25.244767 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:25.244768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:25.244817 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:25.245071 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:25.245076 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:25.245112 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.245162 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.245244 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260620 23:03:25.245464 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.238039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.17.188.126:33935)
0620 23:03:25.238107 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.238109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.238247 (+ 138us) server_negotiation.cc:407] Connection header received
0620 23:03:25.238316 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.238317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.238327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.238391 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:25.238815 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.239561 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.240265 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.240519 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.240589 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.242767 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:25.242769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:25.242769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:25.242776 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:25.244300 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:25.244637 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:25.244638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:25.244639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:25.244700 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:25.244878 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:25.244879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:25.244880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:25.245007 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:25.245077 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.245209 (+ 132us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.245307 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":42}
I20260620 23:03:25.247318 19272 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:55908:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 23:03:25.247560 19272 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:25.248920 19675 tablet_service.cc:1467] Processing CreateTablet for tablet 136450ee2f7d4ef6b6c1ef7ff0e095c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=215021ce80fc4580b66f6e94e5682d6c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:03:25.249078 19675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 136450ee2f7d4ef6b6c1ef7ff0e095c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:25.250553 19779 tablet_bootstrap.cc:492] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: Bootstrap starting.
I20260620 23:03:25.250628 19550 tablet_service.cc:1467] Processing CreateTablet for tablet 136450ee2f7d4ef6b6c1ef7ff0e095c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=215021ce80fc4580b66f6e94e5682d6c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:03:25.250741 19550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 136450ee2f7d4ef6b6c1ef7ff0e095c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:25.250937 19779 tablet_bootstrap.cc:654] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:25.252038 19781 tablet_bootstrap.cc:492] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: Bootstrap starting.
I20260620 23:03:25.252275 19410 tablet_service.cc:1467] Processing CreateTablet for tablet 136450ee2f7d4ef6b6c1ef7ff0e095c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=215021ce80fc4580b66f6e94e5682d6c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:03:25.252394 19410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 136450ee2f7d4ef6b6c1ef7ff0e095c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:25.252445 19781 tablet_bootstrap.cc:654] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:25.252553 19779 tablet_bootstrap.cc:492] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: No bootstrap required, opened a new log
I20260620 23:03:25.252592 19779 ts_tablet_manager.cc:1397] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:25.252717 19779 raft_consensus.cc:348] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } }
I20260620 23:03:25.252779 19779 raft_consensus.cc:374] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:25.252799 19779 raft_consensus.cc:729] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78dc014564e46fc87a01073f3653f98, State: Initialized, Role: FOLLOWER
I20260620 23:03:25.252847 19779 consensus_queue.cc:260] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } }
I20260620 23:03:25.252928 19779 ts_tablet_manager.cc:1428] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:25.253053 19785 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.253140 19785 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:25.253542 19781 tablet_bootstrap.cc:492] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: No bootstrap required, opened a new log
I20260620 23:03:25.253588 19781 ts_tablet_manager.cc:1397] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:03:25.253708 19781 raft_consensus.cc:348] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } }
I20260620 23:03:25.253765 19781 raft_consensus.cc:374] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:25.253785 19781 raft_consensus.cc:729] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1670ec8f6074934a5cd484291f409c2, State: Initialized, Role: FOLLOWER
I20260620 23:03:25.253790 19780 tablet_bootstrap.cc:492] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: Bootstrap starting.
I20260620 23:03:25.253825 19781 consensus_queue.cc:260] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } }
I20260620 23:03:25.253909 19781 ts_tablet_manager.cc:1428] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:25.254029 19786 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.254105 19786 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:25.254171 19780 tablet_bootstrap.cc:654] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:25.255113 19780 tablet_bootstrap.cc:492] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: No bootstrap required, opened a new log
I20260620 23:03:25.255154 19780 ts_tablet_manager.cc:1397] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:25.255295 19780 raft_consensus.cc:348] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } }
I20260620 23:03:25.255354 19780 raft_consensus.cc:374] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:25.255367 19780 raft_consensus.cc:729] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4cbb2f014e4838b9e429d595abf331, State: Initialized, Role: FOLLOWER
I20260620 23:03:25.255422 19780 consensus_queue.cc:260] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } }
I20260620 23:03:25.255533 19780 ts_tablet_manager.cc:1428] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:25.255541 19787 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.255694 19787 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:25.274302 19786 raft_consensus.cc:482] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:25.274389 19786 raft_consensus.cc:504] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } }
I20260620 23:03:25.274662 19786 leader_election.cc:288] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e78dc014564e46fc87a01073f3653f98 (127.17.188.67:33991), 6b4cbb2f014e4838b9e429d595abf331 (127.17.188.65:34681)
I20260620 23:03:25.278465 19776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.275081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:59675 (local address 127.17.188.65:34681)
0620 23:03:25.275119 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.275121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.275219 (+ 98us) server_negotiation.cc:407] Connection header received
0620 23:03:25.275300 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.275301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.275309 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.275339 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:25.275562 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.276009 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.277564 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.278137 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.278220 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.278230 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.278371 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.278407 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:03:25.278501 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.275043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:34681 (local address 127.17.188.66:59675)
0620 23:03:25.275169 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.275179 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.275228 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.275395 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.275396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.275397 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:25.275467 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.275468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.276093 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.276095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.277477 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.277479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.278246 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.278247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.278295 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.278298 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.278363 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.278399 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":56,"thread_start_us":65,"threads_started":1}
I20260620 23:03:25.278753 19430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136450ee2f7d4ef6b6c1ef7ff0e095c1" candidate_uuid: "c1670ec8f6074934a5cd484291f409c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4cbb2f014e4838b9e429d595abf331" is_pre_election: true
I20260620 23:03:25.278825 19430 raft_consensus.cc:2454] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1670ec8f6074934a5cd484291f409c2 in term 0.
I20260620 23:03:25.278966 19502 leader_election.cc:302] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4cbb2f014e4838b9e429d595abf331, c1670ec8f6074934a5cd484291f409c2; no voters:
I20260620 23:03:25.279111 19786 raft_consensus.cc:2781] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:25.279150 19786 raft_consensus.cc:482] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:25.279183 19786 raft_consensus.cc:3037] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:25.279214 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.274876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:37299 (local address 127.17.188.67:33991)
0620 23:03:25.274918 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.274920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.275067 (+ 147us) server_negotiation.cc:407] Connection header received
0620 23:03:25.275195 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.275196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.275210 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.275269 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:25.275435 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.276124 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.278276 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.278798 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.278873 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.278881 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.279088 (+ 207us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.279137 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:25.279258 19820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.274833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33991 (local address 127.17.188.66:37299)
0620 23:03:25.275007 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.275019 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.275095 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.275275 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.275276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.275278 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:25.275349 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.275350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.276204 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.276205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.277478 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.277479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.278915 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.278916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.278993 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.278997 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.279054 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.279090 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":14,"thread_start_us":96,"threads_started":1}
I20260620 23:03:25.279502 19710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136450ee2f7d4ef6b6c1ef7ff0e095c1" candidate_uuid: "c1670ec8f6074934a5cd484291f409c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e78dc014564e46fc87a01073f3653f98" is_pre_election: true
I20260620 23:03:25.279577 19710 raft_consensus.cc:2454] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1670ec8f6074934a5cd484291f409c2 in term 0.
I20260620 23:03:25.280045 19786 raft_consensus.cc:504] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } }
I20260620 23:03:25.280158 19786 leader_election.cc:288] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [CANDIDATE]: Term 1 election: Requested vote from peers e78dc014564e46fc87a01073f3653f98 (127.17.188.67:33991), 6b4cbb2f014e4838b9e429d595abf331 (127.17.188.65:34681)
I20260620 23:03:25.280315 19710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136450ee2f7d4ef6b6c1ef7ff0e095c1" candidate_uuid: "c1670ec8f6074934a5cd484291f409c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e78dc014564e46fc87a01073f3653f98"
I20260620 23:03:25.280349 19430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136450ee2f7d4ef6b6c1ef7ff0e095c1" candidate_uuid: "c1670ec8f6074934a5cd484291f409c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4cbb2f014e4838b9e429d595abf331"
I20260620 23:03:25.280404 19430 raft_consensus.cc:3037] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:25.280393 19710 raft_consensus.cc:3037] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:25.281076 19430 raft_consensus.cc:2454] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1670ec8f6074934a5cd484291f409c2 in term 1.
I20260620 23:03:25.281162 19710 raft_consensus.cc:2454] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1670ec8f6074934a5cd484291f409c2 in term 1.
I20260620 23:03:25.281356 19502 leader_election.cc:302] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1670ec8f6074934a5cd484291f409c2, e78dc014564e46fc87a01073f3653f98; no voters:
I20260620 23:03:25.281509 19786 raft_consensus.cc:2781] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:25.281654 19786 raft_consensus.cc:686] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [term 1 LEADER]: Becoming Leader. State: Replica: c1670ec8f6074934a5cd484291f409c2, State: Running, Role: LEADER
I20260620 23:03:25.281714 19786 consensus_queue.cc:237] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } }
I20260620 23:03:25.282235 19822 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: TxnStatusTablet state changed. Reason: New leader c1670ec8f6074934a5cd484291f409c2. Latest consensus state: current_term: 1 leader_uuid: "c1670ec8f6074934a5cd484291f409c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.282320 19822 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:03:25.282285 19271 catalog_manager.cc:5495] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1670ec8f6074934a5cd484291f409c2 (127.17.188.66). New cstate: current_term: 1 leader_uuid: "c1670ec8f6074934a5cd484291f409c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:25.282624 19824 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 23:03:25.282747 19824 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 23:03:25.294325 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.291190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:41759 (local address 127.0.0.1:56634)
0620 23:03:25.291259 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.291269 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.291315 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.291474 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.291474 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.291475 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:03:25.291517 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.291519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.292401 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.292402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.293295 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.293297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.293784 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.293785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.293850 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.293865 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:03:25.294143 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:03:25.294150 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.294194 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.294234 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 23:03:25.294327 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.291233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.17.188.66:41759)
0620 23:03:25.291269 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.291271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.291298 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:03:25.291378 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.291379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.291396 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.291469 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:25.291572 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.292322 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.293406 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.293699 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.293768 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.293965 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:25.294079 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:25.294151 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.294197 (+ 46us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.294215 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:03:25.295446 19710 raft_consensus.cc:1261] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98 [term 1 FOLLOWER]: Refusing update from remote peer c1670ec8f6074934a5cd484291f409c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:25.295477 19430 raft_consensus.cc:1261] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331 [term 1 FOLLOWER]: Refusing update from remote peer c1670ec8f6074934a5cd484291f409c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:25.295683 19786 consensus_queue.cc:1035] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:25.295789 19822 consensus_queue.cc:1035] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:25.296331 19785 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: TxnStatusTablet state changed. Reason: New leader c1670ec8f6074934a5cd484291f409c2. Latest consensus state: current_term: 1 leader_uuid: "c1670ec8f6074934a5cd484291f409c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.296419 19785 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:25.296784 19787 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: TxnStatusTablet state changed. Reason: New leader c1670ec8f6074934a5cd484291f409c2. Latest consensus state: current_term: 1 leader_uuid: "c1670ec8f6074934a5cd484291f409c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.296870 19787 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:25.297441 19822 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1670ec8f6074934a5cd484291f409c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.297516 19822 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:03:25.297614 19785 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1670ec8f6074934a5cd484291f409c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.297696 19785 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P e78dc014564e46fc87a01073f3653f98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:25.298532 19786 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1670ec8f6074934a5cd484291f409c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.298614 19786 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P c1670ec8f6074934a5cd484291f409c2: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:03:25.298738 19787 tablet_replica.cc:439] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1670ec8f6074934a5cd484291f409c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 } } peers { permanent_uuid: "e78dc014564e46fc87a01073f3653f98" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33991 } } peers { permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 } } }
I20260620 23:03:25.298823 19787 tablet_replica.cc:442] T 136450ee2f7d4ef6b6c1ef7ff0e095c1 P 6b4cbb2f014e4838b9e429d595abf331: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:25.298933 19793 mvcc.cc:205] Tried to move back new op lower bound from 7299058095288193024 to 7299058095234236416. Current Snapshot: MvccSnapshot[applied={T|T < 7299058095288193024 or (T in {7299058095288193024})}]
I20260620 23:03:25.302395 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.299885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.17.188.67:33991)
0620 23:03:25.299923 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.299925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.299934 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:03:25.300025 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.300026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.300043 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.300120 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:25.300194 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.300900 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.301626 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.301914 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.301982 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.302095 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:25.302203 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:25.302269 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.302303 (+ 34us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.302318 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:25.306753 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.304400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:52743 (local address 127.17.188.66:41759)
0620 23:03:25.304441 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.304443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.304511 (+ 68us) server_negotiation.cc:407] Connection header received
0620 23:03:25.304570 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.304570 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.304577 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.304614 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:25.304723 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.305121 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.306165 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.306347 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.306397 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.306567 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:25.306589 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:25.306625 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.306701 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.306713 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:03:25.306821 19831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.304321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:41759 (local address 127.17.188.67:52743)
0620 23:03:25.304456 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.304465 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.304523 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.304625 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.304626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.304627 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:03:25.304671 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.304672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.305183 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.305184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.306087 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.306089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.306417 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.306418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.306478 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.306490 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:03:25.306644 (+ 154us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:03:25.306646 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.306698 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.306738 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":83,"threads_started":1}
I20260620 23:03:25.308957 19430 raft_consensus.cc:1261] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [term 1 FOLLOWER]: Refusing update from remote peer e78dc014564e46fc87a01073f3653f98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:25.308985 19570 raft_consensus.cc:1261] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Refusing update from remote peer e78dc014564e46fc87a01073f3653f98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:25.309212 19809 consensus_queue.cc:1035] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1670ec8f6074934a5cd484291f409c2" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 41759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:25.309347 19785 consensus_queue.cc:1035] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b4cbb2f014e4838b9e429d595abf331" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:25.317704 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.315162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:33819 (local address 127.17.188.67:33991)
0620 23:03:25.315197 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.315199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.315327 (+ 128us) server_negotiation.cc:407] Connection header received
0620 23:03:25.315402 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.315403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.315411 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.315466 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:25.315582 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.315971 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.317088 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.317254 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.317302 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.317509 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:25.317531 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:25.317579 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.317652 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.317664 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:03:25.317778 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.315075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33991 (local address 127.17.188.66:33819)
0620 23:03:25.315264 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:25.315274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:25.315332 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:25.315477 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:25.315477 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:25.315478 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:03:25.315533 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.315534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.316034 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.316035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.316965 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:25.316967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.317349 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.317350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:25.317416 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.317429 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:03:25.317602 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:03:25.317603 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:03:25.317652 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:03:25.317690 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260620 23:03:25.326691 19271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:25.326802 19271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:25.327442 19271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d69f3bde4f44be9a43ab06c8cb6b6b2: Sending DeleteTablet for 3 replicas of tablet 652e87d9edaa4cf0b9460a8e7654e500
I20260620 23:03:25.327678 19410 tablet_service.cc:1514] Processing DeleteTablet for tablet 652e87d9edaa4cf0b9460a8e7654e500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:25 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43436
I20260620 23:03:25.327710 19675 tablet_service.cc:1514] Processing DeleteTablet for tablet 652e87d9edaa4cf0b9460a8e7654e500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:25 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:40256
I20260620 23:03:25.327765 19803 tablet_replica.cc:330] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: stopping tablet replica
I20260620 23:03:25.327808 19803 raft_consensus.cc:2229] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:25.327834 19803 raft_consensus.cc:2258] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:25.327895 19550 tablet_service.cc:1514] Processing DeleteTablet for tablet 652e87d9edaa4cf0b9460a8e7654e500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:25 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56628
I20260620 23:03:25.327957 19802 tablet_replica.cc:330] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: stopping tablet replica
I20260620 23:03:25.328012 19802 raft_consensus.cc:2229] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:25.328038 19802 raft_consensus.cc:2258] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:25.328161 19804 tablet_replica.cc:330] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: stopping tablet replica
I20260620 23:03:25.328219 19804 raft_consensus.cc:2229] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:25.328306 19804 raft_consensus.cc:2258] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:25.328385 19802 ts_tablet_manager.cc:1905] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:25.329998 19802 ts_tablet_manager.cc:1918] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:03:25.330053 19802 log.cc:1196] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/652e87d9edaa4cf0b9460a8e7654e500
I20260620 23:03:25.330086 19803 ts_tablet_manager.cc:1905] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:25.330276 19802 ts_tablet_manager.cc:1939] T 652e87d9edaa4cf0b9460a8e7654e500 P c1670ec8f6074934a5cd484291f409c2: Deleting consensus metadata
I20260620 23:03:25.331544 19260 catalog_manager.cc:4841] TS c1670ec8f6074934a5cd484291f409c2 (127.17.188.66:41759): tablet 652e87d9edaa4cf0b9460a8e7654e500 (table test-table [id=525c8da595e64a2db57b71141ba85282]) successfully deleted
I20260620 23:03:25.331810 19804 ts_tablet_manager.cc:1905] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:25.333287 19804 ts_tablet_manager.cc:1918] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:03:25.333360 19804 log.cc:1196] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/652e87d9edaa4cf0b9460a8e7654e500
I20260620 23:03:25.333612 19804 ts_tablet_manager.cc:1939] T 652e87d9edaa4cf0b9460a8e7654e500 P e78dc014564e46fc87a01073f3653f98: Deleting consensus metadata
Jun 20 23:03:25 dist-test-slave-zpss krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996604, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:25.333904 19258 catalog_manager.cc:4841] TS e78dc014564e46fc87a01073f3653f98 (127.17.188.67:33991): tablet 652e87d9edaa4cf0b9460a8e7654e500 (table test-table [id=525c8da595e64a2db57b71141ba85282]) successfully deleted
I20260620 23:03:25.334414 19803 ts_tablet_manager.cc:1918] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:03:25.334483 19803 log.cc:1196] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/652e87d9edaa4cf0b9460a8e7654e500
I20260620 23:03:25.335047 19803 ts_tablet_manager.cc:1939] T 652e87d9edaa4cf0b9460a8e7654e500 P 6b4cbb2f014e4838b9e429d595abf331: Deleting consensus metadata
I20260620 23:03:25.335688 19258 catalog_manager.cc:4841] TS 6b4cbb2f014e4838b9e429d595abf331 (127.17.188.65:34681): tablet 652e87d9edaa4cf0b9460a8e7654e500 (table test-table [id=525c8da595e64a2db57b71141ba85282]) successfully deleted
I20260620 23:03:25.337500 19776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.328373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.17.188.65:34681)
0620 23:03:25.328409 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.328410 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.328483 (+ 73us) server_negotiation.cc:407] Connection header received
0620 23:03:25.328562 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.328563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.328571 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.328597 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:25.329121 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.329625 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.330616 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.330817 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.330898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.334293 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:25.334301 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:25.334301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:25.334308 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:25.336107 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:25.336469 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:25.336472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:25.336472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:25.336532 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:25.336772 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:25.336772 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:25.336773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:25.336954 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:25.337118 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.337162 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.337237 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:03:25.337703 19470 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43460
I20260620 23:03:25.347606 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:25.338530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.17.188.126:33935)
0620 23:03:25.338568 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:25.338570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:25.341485 (+ 2915us) server_negotiation.cc:407] Connection header received
0620 23:03:25.341555 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:25.341556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:25.341567 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:25.341621 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:25.342099 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.342579 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.343611 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:25.343879 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:25.343922 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:25.345059 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:25.345061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:25.345061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:25.345068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:25.346468 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:25.346828 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:25.346829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:25.346830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:25.346882 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:25.347101 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:25.347102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:25.347102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:25.347217 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:25.347322 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:25.347424 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 23:03:25.347512 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:03:25.347860 19271 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55922
I20260620 23:03:25.348430 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 19345
I20260620 23:03:25.355309 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 19485
I20260620 23:03:25.361112 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 19625
I20260620 23:03:25.365761 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 19241
2026-06-20T23:03:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2388 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:25 dist-test-slave-zpss krb5kdc[19846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:25 dist-test-slave-zpss krb5kdc[19846](info): set up 2 sockets
Jun 20 23:03:25 dist-test-slave-zpss krb5kdc[19846](info): commencing operation
krb5kdc: starting...
W20260620 23:03:26.360818 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:26 dist-test-slave-zpss krb5kdc[19846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:26.465758 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45911
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:34275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:45911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:26.542302 19859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:26.542484 19859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:26.542506 19859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:26.543663 19859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:26.543704 19859 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 23:03:26.543716 19859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:26.543758 19859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:26.543784 19859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:26.543800 19859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:26.545176 19859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:34275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:45911
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:26.545440 19859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:26.545674 19859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:26.550182 19865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:26.550182 19867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:26.550212 19864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:26.550400 19859 server_base.cc:1034] running on GCE node
I20260620 23:03:26.550632 19859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:26.550908 19859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:26.552070 19859 hybrid_clock.cc:648] HybridClock initialized: now 1781996606552055 us; error 41 us; skew 500 ppm
Jun 20 23:03:26 dist-test-slave-zpss krb5kdc[19846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:26.554471 19859 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:26.554947 19859 webserver.cc:460] Webserver started at http://127.17.188.126:38293/ using document root <none> and password file <none>
I20260620 23:03:26.555168 19859 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:26.555234 19859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:26.555373 19859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:26.556355 19859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "a7f6c87a25594d9fb75b14dc3024da4b"
format_stamp: "Formatted at 2026-06-20 23:03:26 on dist-test-slave-zpss"
server_key: "9966b3de315b3d879980a76acbb41780"
server_key_iv: "ca9c4cd835d95e7303d10e8716cf2d7d"
server_key_version: "encryptionkey@0"
I20260620 23:03:26.556725 19859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "a7f6c87a25594d9fb75b14dc3024da4b"
format_stamp: "Formatted at 2026-06-20 23:03:26 on dist-test-slave-zpss"
server_key: "9966b3de315b3d879980a76acbb41780"
server_key_iv: "ca9c4cd835d95e7303d10e8716cf2d7d"
server_key_version: "encryptionkey@0"
I20260620 23:03:26.558071 19859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:26.558727 19874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:26.558920 19859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:26.559019 19859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "a7f6c87a25594d9fb75b14dc3024da4b"
format_stamp: "Formatted at 2026-06-20 23:03:26 on dist-test-slave-zpss"
server_key: "9966b3de315b3d879980a76acbb41780"
server_key_iv: "ca9c4cd835d95e7303d10e8716cf2d7d"
server_key_version: "encryptionkey@0"
I20260620 23:03:26.559087 19859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:26.574277 19859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:26.576120 19859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:26.576279 19859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:26.580760 19859 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:45911
I20260620 23:03:26.580799 19936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:45911 every 8 connection(s)
I20260620 23:03:26.581103 19859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:26.581707 19937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:26.583674 19937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b: Bootstrap starting.
I20260620 23:03:26.584161 19937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:26.584378 19937 log.cc:824] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:26.584932 19937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b: No bootstrap required, opened a new log
I20260620 23:03:26.586948 19937 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6c87a25594d9fb75b14dc3024da4b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45911 } }
I20260620 23:03:26.587071 19937 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:26.587092 19937 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f6c87a25594d9fb75b14dc3024da4b, State: Initialized, Role: FOLLOWER
I20260620 23:03:26.587216 19937 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6c87a25594d9fb75b14dc3024da4b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45911 } }
I20260620 23:03:26.587288 19937 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:26.587311 19937 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:26.587337 19937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:26.587811 19937 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6c87a25594d9fb75b14dc3024da4b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45911 } }
I20260620 23:03:26.587908 19937 leader_election.cc:302] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7f6c87a25594d9fb75b14dc3024da4b; no voters:
I20260620 23:03:26.588043 19937 leader_election.cc:288] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:26.588064 19940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:26.588187 19940 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [term 1 LEADER]: Becoming Leader. State: Replica: a7f6c87a25594d9fb75b14dc3024da4b, State: Running, Role: LEADER
I20260620 23:03:26.588279 19937 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:26.588263 19940 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6c87a25594d9fb75b14dc3024da4b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45911 } }
I20260620 23:03:26.588615 19942 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f6c87a25594d9fb75b14dc3024da4b. Latest consensus state: current_term: 1 leader_uuid: "a7f6c87a25594d9fb75b14dc3024da4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6c87a25594d9fb75b14dc3024da4b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45911 } } }
I20260620 23:03:26.588636 19941 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7f6c87a25594d9fb75b14dc3024da4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6c87a25594d9fb75b14dc3024da4b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45911 } } }
I20260620 23:03:26.588814 19941 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:26.588781 19942 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:26.589154 19948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:26.589735 19948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:26.589975 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 19859
I20260620 23:03:26.590063 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:26.590250 18161 external_mini_cluster.cc:1428] Setting key b34c99f41b7117adb3aa8d40e19e3daa
I20260620 23:03:26.591162 19948 catalog_manager.cc:1347] Generated new cluster ID: d21cd59ab8e04735aacd47066a7056c4
I20260620 23:03:26.591225 19948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:03:26 dist-test-slave-zpss krb5kdc[19846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:26.601802 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:26.590633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.17.188.126:45911)
0620 23:03:26.590784 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:26.590794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:26.590892 (+ 98us) server_negotiation.cc:407] Connection header received
0620 23:03:26.591045 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:26.591052 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:26.591105 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:26.591242 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:26.591995 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.593398 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.594569 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.595200 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.595295 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:26.598113 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:26.598117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:26.598117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:26.598130 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:26.600128 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.600516 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.600519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.600519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.600656 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.600857 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.600857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.600858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.601168 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:26.601367 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:26.601433 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 23:03:26.601589 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260620 23:03:26.617368 19948 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:26.618119 19948 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:26.628293 19948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b: Generated new TSK 0
I20260620 23:03:26.628455 19948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:03:26.658222 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45911
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:34275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:26.736639 19963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:26.736867 19963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:26.736907 19963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:26.738090 19963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:26.738155 19963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:26.738190 19963 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 23:03:26.738217 19963 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:03:26.738250 19963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:26.739718 19963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:34275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.188.126:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:26.739976 19963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:26.740201 19963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:26.744940 19971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:26.744948 19969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:26.744939 19968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:26.745195 19963 server_base.cc:1034] running on GCE node
I20260620 23:03:26.745417 19963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:26.745599 19963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:26.746732 19963 hybrid_clock.cc:648] HybridClock initialized: now 1781996606746716 us; error 30 us; skew 500 ppm
Jun 20 23:03:26 dist-test-slave-zpss krb5kdc[19846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:26.749197 19963 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:03:26.749655 19963 webserver.cc:460] Webserver started at http://127.17.188.65:41619/ using document root <none> and password file <none>
I20260620 23:03:26.749886 19963 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:26.749928 19963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:26.750072 19963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:26.751058 19963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "6595666887684852bb22ecb873075da7"
format_stamp: "Formatted at 2026-06-20 23:03:26 on dist-test-slave-zpss"
server_key: "b4eb97a9cf480becf99b7877213f4f49"
server_key_iv: "03a34f9512678ef92a970be9130bde40"
server_key_version: "encryptionkey@0"
I20260620 23:03:26.751394 19963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "6595666887684852bb22ecb873075da7"
format_stamp: "Formatted at 2026-06-20 23:03:26 on dist-test-slave-zpss"
server_key: "b4eb97a9cf480becf99b7877213f4f49"
server_key_iv: "03a34f9512678ef92a970be9130bde40"
server_key_version: "encryptionkey@0"
I20260620 23:03:26.752697 19963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:26.753445 19978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:26.753609 19963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:26.753686 19963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "6595666887684852bb22ecb873075da7"
format_stamp: "Formatted at 2026-06-20 23:03:26 on dist-test-slave-zpss"
server_key: "b4eb97a9cf480becf99b7877213f4f49"
server_key_iv: "03a34f9512678ef92a970be9130bde40"
server_key_version: "encryptionkey@0"
I20260620 23:03:26.753741 19963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:26.775267 19963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:26.777252 19963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:26.777390 19963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:26.777877 19963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:26.777911 19963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:26.777938 19963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:26.777990 19963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:26.784549 19963 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:45589
I20260620 23:03:26.784610 20093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:45589 every 8 connection(s)
I20260620 23:03:26.784893 19963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:03:26 dist-test-slave-zpss krb5kdc[19846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:26.787901 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:26.778400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:45425 (local address 127.17.188.126:45911)
0620 23:03:26.778472 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:26.778474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:26.778748 (+ 274us) server_negotiation.cc:407] Connection header received
0620 23:03:26.779356 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:26.779358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:26.779376 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:26.779464 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:26.780341 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.781188 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.782773 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.783119 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.783198 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:26.785642 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:26.785643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:26.785643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:26.785649 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:26.786802 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.787142 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.787143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.787143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.787193 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.787391 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.787392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.787392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.787491 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:26.787556 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:26.787775 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 23:03:26.787840 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:03:26.788065 19987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:26.778365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45911 (local address 127.17.188.65:45425)
0620 23:03:26.778632 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:26.778660 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:26.779208 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:26.779574 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:26.779579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:26.779945 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:26.780213 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:26.780219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.781302 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.781303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:26.782684 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:26.782686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.783198 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.783199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:26.783353 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:26.783871 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:26.783899 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:26.785563 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:26.786872 (+ 1309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:26.786878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:26.786879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:26.787078 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:26.787249 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:26.787249 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:26.787250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:26.787339 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:26.787559 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:26.787563 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:26.787663 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 23:03:26.787747 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 23:03:26.787854 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":112,"threads_started":1}
I20260620 23:03:26.788587 20094 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45911
I20260620 23:03:26.788683 20094 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:26.788884 20094 heartbeater.cc:511] Master 127.17.188.126:45911 requested a full tablet report, sending...
I20260620 23:03:26.789500 19890 ts_manager.cc:194] Registered new tserver with Master: 6595666887684852bb22ecb873075da7 (127.17.188.65:45589)
I20260620 23:03:26.790278 19890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:45425
I20260620 23:03:26.792891 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 19963
I20260620 23:03:26.792994 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:26.793179 18161 external_mini_cluster.cc:1428] Setting key 9ec1bd83e56221c6d3b1525d0b156563
I20260620 23:03:26.795289 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:26.789803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:49523 (local address 127.17.188.126:45911)
0620 23:03:26.789842 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:26.789844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:26.789852 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:26.789893 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:26.789894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:26.789904 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:26.789955 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:26.790255 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.790955 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.791614 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.791784 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.791835 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:26.792584 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:26.792585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:26.792585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:26.792591 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:26.794207 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.794550 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.794551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.794552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.794603 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.794802 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.794803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.794803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.794905 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:26.794976 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:26.795147 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 23:03:26.795211 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:26.795306 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:26.789606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45911 (local address 127.17.188.65:49523)
0620 23:03:26.789751 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:26.789762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:26.789840 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:26.789960 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:26.789961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:26.790117 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:26.790172 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:26.790173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.791056 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.791057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:26.791546 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:26.791547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.791847 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.791848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:26.791897 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:26.792248 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:26.792252 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:26.792514 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:26.794285 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:26.794286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:26.794286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:26.794446 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:26.794674 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:26.794675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:26.794676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:26.794731 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:26.794969 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:26.794970 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:26.795022 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 23:03:26.795111 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 23:03:26.795208 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:03:26.815735 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45911
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:34275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:26.894269 20103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:26.894459 20103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:26.894480 20103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:26.895610 20103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:26.895651 20103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:26.895671 20103 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 23:03:26.895728 20103 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:03:26.895756 20103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:26.897168 20103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:34275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.188.126:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:26.897413 20103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:26.897629 20103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:26.902273 20111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:26.902386 20103 server_base.cc:1034] running on GCE node
W20260620 23:03:26.902256 20109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:26.902204 20108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:26.902695 20103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:26.902909 20103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:26.904070 20103 hybrid_clock.cc:648] HybridClock initialized: now 1781996606904057 us; error 26 us; skew 500 ppm
Jun 20 23:03:26 dist-test-slave-zpss krb5kdc[19846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:26.906282 20103 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:03:26.906687 20103 webserver.cc:460] Webserver started at http://127.17.188.66:40417/ using document root <none> and password file <none>
I20260620 23:03:26.906889 20103 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:26.906955 20103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:26.907096 20103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:26.908042 20103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "5c097f779b534501994a894b85f0e2b0"
format_stamp: "Formatted at 2026-06-20 23:03:26 on dist-test-slave-zpss"
server_key: "8a5d468ce2e06c63f458033333438ceb"
server_key_iv: "b3c9c33b209921fb4babbd2d427b4b54"
server_key_version: "encryptionkey@0"
I20260620 23:03:26.908360 20103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "5c097f779b534501994a894b85f0e2b0"
format_stamp: "Formatted at 2026-06-20 23:03:26 on dist-test-slave-zpss"
server_key: "8a5d468ce2e06c63f458033333438ceb"
server_key_iv: "b3c9c33b209921fb4babbd2d427b4b54"
server_key_version: "encryptionkey@0"
I20260620 23:03:26.909562 20103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:26.910219 20118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:26.910377 20103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:26.910465 20103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "5c097f779b534501994a894b85f0e2b0"
format_stamp: "Formatted at 2026-06-20 23:03:26 on dist-test-slave-zpss"
server_key: "8a5d468ce2e06c63f458033333438ceb"
server_key_iv: "b3c9c33b209921fb4babbd2d427b4b54"
server_key_version: "encryptionkey@0"
I20260620 23:03:26.910537 20103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:26.921438 20103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:26.923000 20103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:26.923146 20103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:26.923560 20103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:26.923625 20103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:26.923668 20103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:26.923691 20103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:26.930012 20103 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:46507
I20260620 23:03:26.930090 20233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:46507 every 8 connection(s)
Jun 20 23:03:26 dist-test-slave-zpss krb5kdc[19846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:26.930401 20103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:26.932999 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:26.924118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:59857 (local address 127.17.188.126:45911)
0620 23:03:26.924194 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:26.924196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:26.924382 (+ 186us) server_negotiation.cc:407] Connection header received
0620 23:03:26.924877 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:26.924879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:26.924895 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:26.924967 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:26.925786 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.926627 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.928165 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.928433 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.928516 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:26.930829 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:26.930830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:26.930831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:26.930837 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:26.931951 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.932328 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.932328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.932329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.932377 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.932529 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.932530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.932530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.932616 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:26.932659 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:26.932883 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 23:03:26.932940 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 23:03:26.933108 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:26.924055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45911 (local address 127.17.188.66:59857)
0620 23:03:26.924254 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:26.924290 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:26.924737 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:26.925056 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:26.925061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:26.925425 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:26.925677 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:26.925693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.926753 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.926754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:26.928065 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:26.928067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.928522 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.928524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:26.928656 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:26.929191 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:26.929201 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:26.930727 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:26.932016 (+ 1289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:26.932021 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:26.932022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:26.932263 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:26.932417 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:26.932417 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:26.932418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:26.932494 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:26.932667 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:26.932670 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:26.932766 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 23:03:26.932853 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 23:03:26.932952 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260620 23:03:26.933569 20234 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45911
I20260620 23:03:26.933684 20234 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:26.933838 20234 heartbeater.cc:511] Master 127.17.188.126:45911 requested a full tablet report, sending...
I20260620 23:03:26.934221 19890 ts_manager.cc:194] Registered new tserver with Master: 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507)
I20260620 23:03:26.934664 19890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:59857
I20260620 23:03:26.939234 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:26.934563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:49235 (local address 127.17.188.126:45911)
0620 23:03:26.934605 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:26.934607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:26.934661 (+ 54us) server_negotiation.cc:407] Connection header received
0620 23:03:26.934748 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:26.934749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:26.934759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:26.934829 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:26.935176 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.935602 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.936206 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.936374 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.936442 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:26.937265 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:26.937265 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:26.937266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:26.937270 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:26.938400 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.938682 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.938683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.938683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.938727 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:26.938862 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:26.938863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:26.938863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:26.938987 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:26.939030 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:26.939125 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 23:03:26.939174 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:26.939240 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:26.934492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45911 (local address 127.17.188.66:49235)
0620 23:03:26.934608 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:26.934618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:26.934674 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:26.934835 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:26.934836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:26.935045 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:26.935103 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:26.935104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.935686 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.935687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:26.936133 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:26.936135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:26.936455 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:26.936456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:26.936506 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:26.936932 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:26.936937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:26.937199 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:26.938466 (+ 1267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:26.938467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:26.938467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:26.938622 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:26.938779 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:26.938779 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:26.938780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:26.938826 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:26.939034 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:26.939034 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:26.939065 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 23:03:26.939112 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 23:03:26.939178 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260620 23:03:26.940078 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 20103
I20260620 23:03:26.940214 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:26.940418 18161 external_mini_cluster.cc:1428] Setting key a0776ca6c8ca4649de7229191969a6c1
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:03:26.962082 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45911
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:34275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:27.039049 20243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:27.039263 20243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:27.039299 20243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:27.040355 20243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:27.040417 20243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:27.040453 20243 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 23:03:27.040479 20243 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:03:27.040510 20243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:27.041898 20243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:34275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.188.126:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:27.042152 20243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:27.042364 20243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:03:27.047214 20243 server_base.cc:1034] running on GCE node
W20260620 23:03:27.047190 20249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:27.047271 20248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:27.047333 20251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:27.047549 20243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:27.047814 20243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:27.048959 20243 hybrid_clock.cc:648] HybridClock initialized: now 1781996607048951 us; error 25 us; skew 500 ppm
Jun 20 23:03:27 dist-test-slave-zpss krb5kdc[19846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996607, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:27.051282 20243 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:03:27.051687 20243 webserver.cc:460] Webserver started at http://127.17.188.67:44701/ using document root <none> and password file <none>
I20260620 23:03:27.051859 20243 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:27.051896 20243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:27.051978 20243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:27.052920 20243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c"
format_stamp: "Formatted at 2026-06-20 23:03:27 on dist-test-slave-zpss"
server_key: "d320c90008beb6b7f70cd40d2a862abd"
server_key_iv: "b20a048288e7aa95f85406e908417037"
server_key_version: "encryptionkey@0"
I20260620 23:03:27.053215 20243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c"
format_stamp: "Formatted at 2026-06-20 23:03:27 on dist-test-slave-zpss"
server_key: "d320c90008beb6b7f70cd40d2a862abd"
server_key_iv: "b20a048288e7aa95f85406e908417037"
server_key_version: "encryptionkey@0"
I20260620 23:03:27.054524 20243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:27.055152 20258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:27.055327 20243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:27.055413 20243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c"
format_stamp: "Formatted at 2026-06-20 23:03:27 on dist-test-slave-zpss"
server_key: "d320c90008beb6b7f70cd40d2a862abd"
server_key_iv: "b20a048288e7aa95f85406e908417037"
server_key_version: "encryptionkey@0"
I20260620 23:03:27.055500 20243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:27.072208 20243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:27.074065 20243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:27.074234 20243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:27.074784 20243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:27.074843 20243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:27.074900 20243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:27.074923 20243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:03:27 dist-test-slave-zpss krb5kdc[19846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996607, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:27.082265 20243 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:40293
I20260620 23:03:27.082626 20243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:27.082669 20373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:40293 every 8 connection(s)
I20260620 23:03:27.085144 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.075211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:38227 (local address 127.17.188.126:45911)
0620 23:03:27.075288 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.075292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.075514 (+ 222us) server_negotiation.cc:407] Connection header received
0620 23:03:27.075973 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.075975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.075989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.076081 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:27.076907 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.077811 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.078873 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.079229 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.079318 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.082312 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:27.082314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:27.082314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:27.082323 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:27.083899 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:27.084266 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:27.084266 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:27.084267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:27.084315 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:27.084511 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:27.084512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:27.084512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:27.084606 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:27.084687 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.085021 (+ 334us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.085079 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 23:03:27.085410 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.075207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45911 (local address 127.17.188.67:38227)
0620 23:03:27.075388 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.075425 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.075881 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.076136 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.076138 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.076452 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:27.076729 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.076736 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.077910 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.077911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.078785 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.078788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.079363 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.079364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.079490 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.079972 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:27.079983 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:27.082192 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:27.083974 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:27.083979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:27.083980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:27.084201 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:27.084370 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:27.084370 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:27.084370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:27.084459 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:27.084787 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:27.084793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:27.084903 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.084995 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.085110 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":121,"threads_started":1}
I20260620 23:03:27.085909 20374 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45911
I20260620 23:03:27.085963 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 20243
I20260620 23:03:27.085999 20374 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:27.086043 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:27.086181 20374 heartbeater.cc:511] Master 127.17.188.126:45911 requested a full tablet report, sending...
I20260620 23:03:27.086256 18161 external_mini_cluster.cc:1428] Setting key f90ae32a22949c9ddd26fe2700ac0097
I20260620 23:03:27.086552 19890 ts_manager.cc:194] Registered new tserver with Master: f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293)
I20260620 23:03:27.087051 19890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:38227
I20260620 23:03:27.087785 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:27.093199 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.086901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:52921 (local address 127.17.188.126:45911)
0620 23:03:27.086940 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.086942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.087055 (+ 113us) server_negotiation.cc:407] Connection header received
0620 23:03:27.087126 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.087127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.087144 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.087190 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:27.087566 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.088298 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.089380 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.089747 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.089816 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.090792 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:27.090793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:27.090793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:27.090800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:27.092046 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:27.092391 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:27.092392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:27.092393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:27.092446 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:27.092667 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:27.092668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:27.092668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:27.092772 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:27.092845 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.092994 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.093096 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:27.093199 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.086832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45911 (local address 127.17.188.67:52921)
0620 23:03:27.086983 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.087000 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.087070 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.087230 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.087230 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.087418 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:27.087499 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.087500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.088412 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.088413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.089258 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.089260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.089875 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.089877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.089987 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.090433 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:27.090437 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:27.090704 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:27.092140 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:27.092141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:27.092142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:27.092317 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:27.092519 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:27.092520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:27.092520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:27.092577 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:27.092838 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:27.092839 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:27.092882 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.092971 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.093073 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
Jun 20 23:03:27 dist-test-slave-zpss krb5kdc[19846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:03:27 dist-test-slave-zpss krb5kdc[19846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:27.109787 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.103655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.17.188.126:45911)
0620 23:03:27.103717 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.103719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.103728 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:03:27.103773 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.103774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.103782 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.103837 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:27.104319 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.104782 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.105549 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.105724 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.105782 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.107819 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:27.107820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:27.107820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:27.107826 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:27.108877 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:27.109162 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:27.109163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:27.109163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:27.109206 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:27.109415 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:27.109416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:27.109416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:27.109504 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:27.109574 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.109654 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.109711 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:03:27.111409 19891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:27.112888 19891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:27.118919 20389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.115180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:40293 (local address 127.0.0.1:56992)
0620 23:03:27.115422 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.115462 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.115525 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.115732 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.115735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.115737 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:27.115794 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.115795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.116762 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.116763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.117772 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.117775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.118684 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.118685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.118770 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.118789 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.118825 (+ 36us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.118860 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":72,"threads_started":1}
I20260620 23:03:27.119310 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.115330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56992 (local address 127.17.188.67:40293)
0620 23:03:27.115508 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.115512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.115524 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:03:27.115617 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.115618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.115630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.115720 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:27.115881 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.116676 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.117887 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.118583 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.119120 (+ 537us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.119183 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.119209 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.119244 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260620 23:03:27.119335 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.115184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:46507 (local address 127.0.0.1:54248)
0620 23:03:27.115648 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.115657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.115696 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.116144 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.116145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.116146 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:27.116177 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.116179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.117437 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.117438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.118320 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.118322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.119143 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.119144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.119202 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.119205 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.119234 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.119270 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":74,"threads_started":1}
I20260620 23:03:27.119680 20388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.115181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45589 (local address 127.0.0.1:33280)
0620 23:03:27.115762 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.115768 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.115820 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.115997 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.115998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.116000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:27.116046 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.116047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.117381 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.117382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.118630 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.118633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.119500 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.119501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.119556 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.119559 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.119587 (+ 28us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.119617 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":151,"thread_start_us":49,"threads_started":1}
I20260620 23:03:27.119706 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.115449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.17.188.66:46507)
0620 23:03:27.115931 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.115935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.115943 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:27.115978 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.115980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.115993 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.116244 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:27.116297 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.117323 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.118401 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.119048 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.119482 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.119538 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.119560 (+ 22us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.119601 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":72,"threads_started":1}
I20260620 23:03:27.119931 20308 tablet_service.cc:1467] Processing CreateTablet for tablet 404c5809014a4d96a933542e6bacf44a (DEFAULT_TABLE table=test-table [id=571749c12bf04355bcf3b94a7faa0abc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:27.120048 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.115515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.17.188.65:45589)
0620 23:03:27.115680 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.115684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.115821 (+ 137us) server_negotiation.cc:407] Connection header received
0620 23:03:27.115883 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.115886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.115900 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.116121 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:27.116185 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.117280 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.118709 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.119411 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.119853 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.119910 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.119932 (+ 22us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.119963 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260620 23:03:27.120311 20308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404c5809014a4d96a933542e6bacf44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:27.120291 20168 tablet_service.cc:1467] Processing CreateTablet for tablet 404c5809014a4d96a933542e6bacf44a (DEFAULT_TABLE table=test-table [id=571749c12bf04355bcf3b94a7faa0abc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:27.120617 20168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404c5809014a4d96a933542e6bacf44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:27.120626 20028 tablet_service.cc:1467] Processing CreateTablet for tablet 404c5809014a4d96a933542e6bacf44a (DEFAULT_TABLE table=test-table [id=571749c12bf04355bcf3b94a7faa0abc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:27.120959 20028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404c5809014a4d96a933542e6bacf44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:27.123376 20394 tablet_bootstrap.cc:492] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: Bootstrap starting.
I20260620 23:03:27.123901 20395 tablet_bootstrap.cc:492] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: Bootstrap starting.
I20260620 23:03:27.124035 20394 tablet_bootstrap.cc:654] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:27.124349 20396 tablet_bootstrap.cc:492] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: Bootstrap starting.
I20260620 23:03:27.124528 20394 log.cc:824] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:27.124778 20395 tablet_bootstrap.cc:654] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:27.125010 20396 tablet_bootstrap.cc:654] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:27.125100 20395 log.cc:824] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:27.125382 20394 tablet_bootstrap.cc:492] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: No bootstrap required, opened a new log
I20260620 23:03:27.125407 20396 log.cc:824] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:27.125460 20394 ts_tablet_manager.cc:1397] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:03:27.125854 20395 tablet_bootstrap.cc:492] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: No bootstrap required, opened a new log
I20260620 23:03:27.125926 20395 ts_tablet_manager.cc:1397] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:27.126070 20396 tablet_bootstrap.cc:492] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: No bootstrap required, opened a new log
I20260620 23:03:27.126137 20396 ts_tablet_manager.cc:1397] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:03:27.127415 20394 raft_consensus.cc:348] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } }
I20260620 23:03:27.127576 20394 raft_consensus.cc:374] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:27.127610 20394 raft_consensus.cc:729] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c097f779b534501994a894b85f0e2b0, State: Initialized, Role: FOLLOWER
I20260620 23:03:27.127616 20396 raft_consensus.cc:348] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } }
I20260620 23:03:27.127746 20394 consensus_queue.cc:260] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } }
I20260620 23:03:27.127785 20396 raft_consensus.cc:374] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:27.127827 20396 raft_consensus.cc:729] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6595666887684852bb22ecb873075da7, State: Initialized, Role: FOLLOWER
I20260620 23:03:27.127974 20396 consensus_queue.cc:260] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } }
I20260620 23:03:27.128072 20394 ts_tablet_manager.cc:1428] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:27.128038 20395 raft_consensus.cc:348] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } }
I20260620 23:03:27.128182 20395 raft_consensus.cc:374] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:27.128211 20234 heartbeater.cc:503] Master 127.17.188.126:45911 was elected leader, sending a full tablet report...
I20260620 23:03:27.128221 20396 ts_tablet_manager.cc:1428] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 23:03:27.128221 20395 raft_consensus.cc:729] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97b009ab6e14dfc8f8ba2660cb3f74c, State: Initialized, Role: FOLLOWER
I20260620 23:03:27.128333 20094 heartbeater.cc:503] Master 127.17.188.126:45911 was elected leader, sending a full tablet report...
I20260620 23:03:27.128352 20395 consensus_queue.cc:260] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } }
I20260620 23:03:27.128592 20395 ts_tablet_manager.cc:1428] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:27.128711 20374 heartbeater.cc:503] Master 127.17.188.126:45911 was elected leader, sending a full tablet report...
W20260620 23:03:27.180979 20235 tablet.cc:2369] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:03:27.285569 20095 tablet.cc:2369] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:27.331955 20402 raft_consensus.cc:482] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:27.332074 20402 raft_consensus.cc:504] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } }
I20260620 23:03:27.332444 20402 leader_election.cc:288] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6595666887684852bb22ecb873075da7 (127.17.188.65:45589), 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507)
W20260620 23:03:27.333201 20375 tablet.cc:2369] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:27.335289 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.332667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:39961 (local address 127.17.188.65:45589)
0620 23:03:27.332714 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.332716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.332744 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:03:27.332811 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.332812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.332824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.332889 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:27.333100 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.333757 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.334616 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.334945 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.334997 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.335008 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.335170 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.335211 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:03:27.335294 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.332625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45589 (local address 127.17.188.67:39961)
0620 23:03:27.332689 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.332701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.332744 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.332927 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.332928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.332930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:27.333005 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.333007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.333829 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.333830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.334534 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.334536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.335019 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.335020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.335101 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.335104 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.335148 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.335196 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 23:03:27.335304 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.332725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:42679 (local address 127.17.188.66:46507)
0620 23:03:27.332767 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.332769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.332888 (+ 119us) server_negotiation.cc:407] Connection header received
0620 23:03:27.332935 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.332936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.332945 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.332997 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:27.333171 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.333806 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.334626 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.334966 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.335050 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.335057 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.335209 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.335247 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:03:27.335317 20403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.332635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:46507 (local address 127.17.188.67:42679)
0620 23:03:27.332809 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.332819 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.332879 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.333005 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.333005 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.333007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:27.333097 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.333098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.333877 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.333878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.334532 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.334534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.335067 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.335068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.335121 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.335124 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.335190 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.335220 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":19,"thread_start_us":91,"threads_started":1}
I20260620 23:03:27.335623 20048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404c5809014a4d96a933542e6bacf44a" candidate_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6595666887684852bb22ecb873075da7" is_pre_election: true
I20260620 23:03:27.335778 20048 raft_consensus.cc:2454] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f97b009ab6e14dfc8f8ba2660cb3f74c in term 0.
I20260620 23:03:27.335796 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404c5809014a4d96a933542e6bacf44a" candidate_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c097f779b534501994a894b85f0e2b0" is_pre_election: true
I20260620 23:03:27.335947 20188 raft_consensus.cc:2454] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f97b009ab6e14dfc8f8ba2660cb3f74c in term 0.
I20260620 23:03:27.336010 20261 leader_election.cc:302] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6595666887684852bb22ecb873075da7, f97b009ab6e14dfc8f8ba2660cb3f74c; no voters:
I20260620 23:03:27.336158 20402 raft_consensus.cc:2781] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:27.336211 20402 raft_consensus.cc:482] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:27.336248 20402 raft_consensus.cc:3037] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:27.337148 20402 raft_consensus.cc:504] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } }
I20260620 23:03:27.337296 20402 leader_election.cc:288] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [CANDIDATE]: Term 1 election: Requested vote from peers 6595666887684852bb22ecb873075da7 (127.17.188.65:45589), 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507)
I20260620 23:03:27.337460 20048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404c5809014a4d96a933542e6bacf44a" candidate_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6595666887684852bb22ecb873075da7"
I20260620 23:03:27.337459 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404c5809014a4d96a933542e6bacf44a" candidate_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c097f779b534501994a894b85f0e2b0"
I20260620 23:03:27.337538 20048 raft_consensus.cc:3037] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:27.337561 20188 raft_consensus.cc:3037] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:27.338233 20048 raft_consensus.cc:2454] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f97b009ab6e14dfc8f8ba2660cb3f74c in term 1.
I20260620 23:03:27.338260 20188 raft_consensus.cc:2454] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f97b009ab6e14dfc8f8ba2660cb3f74c in term 1.
I20260620 23:03:27.338416 20261 leader_election.cc:302] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6595666887684852bb22ecb873075da7, f97b009ab6e14dfc8f8ba2660cb3f74c; no voters:
I20260620 23:03:27.338522 20402 raft_consensus.cc:2781] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:27.338655 20402 raft_consensus.cc:686] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 LEADER]: Becoming Leader. State: Replica: f97b009ab6e14dfc8f8ba2660cb3f74c, State: Running, Role: LEADER
I20260620 23:03:27.338754 20402 consensus_queue.cc:237] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } }
I20260620 23:03:27.339540 19889 catalog_manager.cc:5495] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c reported cstate change: term changed from 0 to 1, leader changed from <none> to f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67). New cstate: current_term: 1 leader_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } health_report { overall_health: UNKNOWN } } }
Jun 20 23:03:27 dist-test-slave-zpss krb5kdc[19846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:27.372413 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.365826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45911 (local address 127.0.0.1:48834)
0620 23:03:27.365987 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.365996 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.366045 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.366189 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.366190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.366452 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:27.366491 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.366493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.367057 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.367057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.367569 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.367570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.367874 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.367874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.367911 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.368366 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:27.368370 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:27.369690 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:27.371291 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:27.371298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:27.371300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:27.371490 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:27.371702 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:27.371702 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:27.371703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:27.371771 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:27.371999 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:27.372006 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:27.372059 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.372145 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.372285 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260620 23:03:27.372418 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.365904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.17.188.126:45911)
0620 23:03:27.365960 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.365962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.366044 (+ 82us) server_negotiation.cc:407] Connection header received
0620 23:03:27.366109 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.366109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.366118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.366167 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:27.366550 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.366995 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.367643 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.367812 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.367861 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.369801 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:27.369803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:27.369803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:27.369810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:27.371167 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:27.371579 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:27.371580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:27.371581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:27.371635 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:27.371834 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:27.371835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:27.371835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:27.371935 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:27.372007 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.372166 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.372336 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":34}
I20260620 23:03:27.374400 19889 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:48834:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 23:03:27.374610 19889 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:27.375973 20308 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6779b1915b49ec85a28bf3f10aa139 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7612b0bda1446c1980a714c844be020]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:03:27.375972 20168 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6779b1915b49ec85a28bf3f10aa139 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7612b0bda1446c1980a714c844be020]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:03:27.375998 20028 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6779b1915b49ec85a28bf3f10aa139 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7612b0bda1446c1980a714c844be020]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:03:27.376124 20308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6779b1915b49ec85a28bf3f10aa139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:27.376123 20168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6779b1915b49ec85a28bf3f10aa139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:27.376173 20028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6779b1915b49ec85a28bf3f10aa139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:27.377352 20396 tablet_bootstrap.cc:492] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: Bootstrap starting.
I20260620 23:03:27.377538 20395 tablet_bootstrap.cc:492] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: Bootstrap starting.
I20260620 23:03:27.377755 20396 tablet_bootstrap.cc:654] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:27.377792 20394 tablet_bootstrap.cc:492] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: Bootstrap starting.
I20260620 23:03:27.377954 20395 tablet_bootstrap.cc:654] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:27.378473 20394 tablet_bootstrap.cc:654] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:27.378574 20395 tablet_bootstrap.cc:492] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: No bootstrap required, opened a new log
I20260620 23:03:27.378629 20395 ts_tablet_manager.cc:1397] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:27.378716 20396 tablet_bootstrap.cc:492] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: No bootstrap required, opened a new log
I20260620 23:03:27.378762 20396 ts_tablet_manager.cc:1397] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:27.378783 20395 raft_consensus.cc:348] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.378854 20395 raft_consensus.cc:374] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:27.378872 20395 raft_consensus.cc:729] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97b009ab6e14dfc8f8ba2660cb3f74c, State: Initialized, Role: FOLLOWER
I20260620 23:03:27.378906 20396 raft_consensus.cc:348] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.378953 20396 raft_consensus.cc:374] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:27.378937 20395 consensus_queue.cc:260] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.378968 20396 raft_consensus.cc:729] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6595666887684852bb22ecb873075da7, State: Initialized, Role: FOLLOWER
I20260620 23:03:27.379019 20395 ts_tablet_manager.cc:1428] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:27.379022 20396 consensus_queue.cc:260] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.379050 20402 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.379103 20402 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:27.379112 20396 ts_tablet_manager.cc:1428] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:27.379148 20401 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.379241 20401 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:27.379647 20394 tablet_bootstrap.cc:492] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: No bootstrap required, opened a new log
I20260620 23:03:27.379721 20394 ts_tablet_manager.cc:1397] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:27.379873 20394 raft_consensus.cc:348] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.379941 20394 raft_consensus.cc:374] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:27.379972 20394 raft_consensus.cc:729] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c097f779b534501994a894b85f0e2b0, State: Initialized, Role: FOLLOWER
I20260620 23:03:27.380025 20394 consensus_queue.cc:260] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.380125 20394 ts_tablet_manager.cc:1428] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:27.380200 20400 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.380321 20400 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:27.446309 20401 raft_consensus.cc:482] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:27.446393 20401 raft_consensus.cc:504] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.446674 20401 leader_election.cc:288] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507), f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293)
I20260620 23:03:27.449476 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.446930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:53735 (local address 127.17.188.66:46507)
0620 23:03:27.446977 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.446979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.447128 (+ 149us) server_negotiation.cc:407] Connection header received
0620 23:03:27.447185 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.447186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.447194 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.447243 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:27.447425 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.447868 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.448730 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.449075 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.449155 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.449165 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.449338 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.449378 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:03:27.449477 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.446858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:46507 (local address 127.17.188.65:53735)
0620 23:03:27.447037 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.447050 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.447121 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.447267 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.447268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.447270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:27.447356 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.447358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.447975 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.447976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.448647 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.448649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.449175 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.449176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.449250 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.449254 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.449295 (+ 41us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.449375 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":41,"thread_start_us":55,"threads_started":1}
I20260620 23:03:27.449870 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6779b1915b49ec85a28bf3f10aa139" candidate_uuid: "6595666887684852bb22ecb873075da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c097f779b534501994a894b85f0e2b0" is_pre_election: true
I20260620 23:03:27.449931 20416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.446858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:40293 (local address 127.17.188.65:52989)
0620 23:03:27.446990 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.446999 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.447048 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.447187 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.447188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.447190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:27.447267 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.447270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.447934 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.447935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.448646 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.448648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.449180 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.449181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.449763 (+ 582us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.449767 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.449827 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.449865 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":55,"threads_started":1}
I20260620 23:03:27.449958 20188 raft_consensus.cc:2454] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6595666887684852bb22ecb873075da7 in term 0.
I20260620 23:03:27.449966 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.446947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:52989 (local address 127.17.188.67:40293)
0620 23:03:27.446987 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.446988 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.447061 (+ 73us) server_negotiation.cc:407] Connection header received
0620 23:03:27.447093 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.447094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.447104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.447165 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:27.447350 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.447839 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.448733 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.449075 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.449155 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.449165 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.449865 (+ 700us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.449906 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:03:27.450232 19982 leader_election.cc:302] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c097f779b534501994a894b85f0e2b0, 6595666887684852bb22ecb873075da7; no voters:
I20260620 23:03:27.450381 20401 raft_consensus.cc:2781] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:27.450314 20328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6779b1915b49ec85a28bf3f10aa139" candidate_uuid: "6595666887684852bb22ecb873075da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" is_pre_election: true
I20260620 23:03:27.450424 20401 raft_consensus.cc:482] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:27.450449 20401 raft_consensus.cc:3037] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:27.450450 20328 raft_consensus.cc:2454] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6595666887684852bb22ecb873075da7 in term 0.
I20260620 23:03:27.451113 20401 raft_consensus.cc:504] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.451270 20401 leader_election.cc:288] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [CANDIDATE]: Term 1 election: Requested vote from peers 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507), f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293)
I20260620 23:03:27.451421 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6779b1915b49ec85a28bf3f10aa139" candidate_uuid: "6595666887684852bb22ecb873075da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c097f779b534501994a894b85f0e2b0"
I20260620 23:03:27.451436 20328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6779b1915b49ec85a28bf3f10aa139" candidate_uuid: "6595666887684852bb22ecb873075da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c"
I20260620 23:03:27.451491 20328 raft_consensus.cc:3037] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:27.451514 20188 raft_consensus.cc:3037] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:27.452059 20188 raft_consensus.cc:2454] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6595666887684852bb22ecb873075da7 in term 1.
I20260620 23:03:27.452210 20328 raft_consensus.cc:2454] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6595666887684852bb22ecb873075da7 in term 1.
I20260620 23:03:27.452298 19982 leader_election.cc:302] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c097f779b534501994a894b85f0e2b0, 6595666887684852bb22ecb873075da7; no voters:
I20260620 23:03:27.452397 20401 raft_consensus.cc:2781] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:27.452530 20401 raft_consensus.cc:686] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [term 1 LEADER]: Becoming Leader. State: Replica: 6595666887684852bb22ecb873075da7, State: Running, Role: LEADER
I20260620 23:03:27.452643 20401 consensus_queue.cc:237] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.453063 20417 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: TxnStatusTablet state changed. Reason: New leader 6595666887684852bb22ecb873075da7. Latest consensus state: current_term: 1 leader_uuid: "6595666887684852bb22ecb873075da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.453145 20417 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:03:27.453307 20419 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 23:03:27.453470 20419 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 23:03:27.453410 19889 catalog_manager.cc:5495] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6595666887684852bb22ecb873075da7 (127.17.188.65). New cstate: current_term: 1 leader_uuid: "6595666887684852bb22ecb873075da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:27.473150 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.470769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33294 (local address 127.17.188.65:45589)
0620 23:03:27.470806 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.470808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.470852 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:03:27.470947 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.470948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.470963 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.471026 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:27.471144 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.471606 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.472374 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.472574 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.472648 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.472842 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.472938 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.473026 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.473069 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.473087 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:03:27.473181 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.470707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45589 (local address 127.0.0.1:33294)
0620 23:03:27.470804 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.470816 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.470874 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.471035 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.471036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.471037 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:03:27.471081 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.471084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.471689 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.471690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.472260 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.472273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.472647 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.472648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.472711 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.472726 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.473006 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.473015 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.473064 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.473107 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260620 23:03:27.474747 20328 raft_consensus.cc:1261] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Refusing update from remote peer 6595666887684852bb22ecb873075da7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:27.474756 20188 raft_consensus.cc:1261] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Refusing update from remote peer 6595666887684852bb22ecb873075da7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:27.474993 20401 consensus_queue.cc:1035] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:27.475210 20401 consensus_queue.cc:1035] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:27.476102 20400 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: TxnStatusTablet state changed. Reason: New leader 6595666887684852bb22ecb873075da7. Latest consensus state: current_term: 1 leader_uuid: "6595666887684852bb22ecb873075da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.476200 20400 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:27.476301 20402 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: TxnStatusTablet state changed. Reason: New leader 6595666887684852bb22ecb873075da7. Latest consensus state: current_term: 1 leader_uuid: "6595666887684852bb22ecb873075da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.476383 20402 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:27.476944 20401 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6595666887684852bb22ecb873075da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.477036 20401 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:03:27.477053 20402 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6595666887684852bb22ecb873075da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.477134 20402 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P f97b009ab6e14dfc8f8ba2660cb3f74c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:27.477298 20422 mvcc.cc:205] Tried to move back new op lower bound from 7299058104214278144 to 7299058104126525440. Current Snapshot: MvccSnapshot[applied={T|T < 7299058104214278144}]
I20260620 23:03:27.478071 20424 mvcc.cc:205] Tried to move back new op lower bound from 7299058104214278144 to 7299058104126525440. Current Snapshot: MvccSnapshot[applied={T|T < 7299058104214278144}]
I20260620 23:03:27.478374 20417 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6595666887684852bb22ecb873075da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.478451 20417 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P 6595666887684852bb22ecb873075da7: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:03:27.478811 20423 mvcc.cc:205] Tried to move back new op lower bound from 7299058104214278144 to 7299058104126525440. Current Snapshot: MvccSnapshot[applied={T|T < 7299058104214278144}]
I20260620 23:03:27.478896 20400 tablet_replica.cc:439] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6595666887684852bb22ecb873075da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } } }
I20260620 23:03:27.478961 20400 tablet_replica.cc:442] T 2c6779b1915b49ec85a28bf3f10aa139 P 5c097f779b534501994a894b85f0e2b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:03:27.482074 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.479884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.17.188.67:40293)
0620 23:03:27.479926 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.479928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.479935 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:03:27.479962 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.479962 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.479977 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.480033 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:27.480160 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.480582 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.481309 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.481532 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.481612 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.481771 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.481865 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.481949 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.481989 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.482007 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:03:27.486931 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.484179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45589 (local address 127.17.188.67:33761)
0620 23:03:27.484258 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.484268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.484311 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.484541 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.484541 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.484542 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:03:27.484583 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.484583 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.485445 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.485446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.486062 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.486064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.486507 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.486507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.486578 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.486592 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.486802 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.486804 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.486848 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.486888 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 23:03:27.486937 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.484364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:33761 (local address 127.17.188.65:45589)
0620 23:03:27.484423 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.484425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.484435 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:03:27.484456 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.484457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.484468 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.484525 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:27.484635 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.485367 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.486141 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.486426 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.486503 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.486700 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.486732 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.486794 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.486851 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.486870 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:03:27.489338 20188 raft_consensus.cc:1261] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Refusing update from remote peer f97b009ab6e14dfc8f8ba2660cb3f74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:27.489367 20048 raft_consensus.cc:1261] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Refusing update from remote peer f97b009ab6e14dfc8f8ba2660cb3f74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:27.489620 20404 consensus_queue.cc:1035] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:27.489795 20404 consensus_queue.cc:1035] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:27.500134 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.497339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:55221 (local address 127.17.188.67:40293)
0620 23:03:27.497372 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.497374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.497448 (+ 74us) server_negotiation.cc:407] Connection header received
0620 23:03:27.497519 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.497520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.497531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.497598 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:27.497775 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.498520 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.499393 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.499666 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.499732 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.499935 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.499964 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.500040 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.500055 (+ 15us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.500073 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:03:27.500201 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.497165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:40293 (local address 127.17.188.65:55221)
0620 23:03:27.497393 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.497404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.497464 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.497613 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.497614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.497632 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:03:27.497684 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.497690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.498607 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.498608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.499325 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.499328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.499759 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.499759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.499838 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.499851 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.500015 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.500017 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.500046 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.500081 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260620 23:03:27.509639 19890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:27.509734 19890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:27.510430 19890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b: Sending DeleteTablet for 3 replicas of tablet 404c5809014a4d96a933542e6bacf44a
I20260620 23:03:27.510744 20308 tablet_service.cc:1514] Processing DeleteTablet for tablet 404c5809014a4d96a933542e6bacf44a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:27 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56992
I20260620 23:03:27.510895 20168 tablet_service.cc:1514] Processing DeleteTablet for tablet 404c5809014a4d96a933542e6bacf44a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:27 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:54248
I20260620 23:03:27.510982 20028 tablet_service.cc:1514] Processing DeleteTablet for tablet 404c5809014a4d96a933542e6bacf44a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:27 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:33280
I20260620 23:03:27.511214 20443 tablet_replica.cc:330] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: stopping tablet replica
I20260620 23:03:27.511376 20443 raft_consensus.cc:2229] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:27.511482 20443 raft_consensus.cc:2258] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:27.511482 19890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:27.511627 19890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:27.511932 20443 ts_tablet_manager.cc:1905] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:27.513269 20168 tablet_service.cc:1467] Processing CreateTablet for tablet 7af7a6c856d647af952f9d98fa302835 (DEFAULT_TABLE table=test-table [id=bcb1f45fb0c24729bff32ea9db93da52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:27.513427 20168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7af7a6c856d647af952f9d98fa302835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:27.514154 20443 ts_tablet_manager.cc:1918] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:03:27.514292 20443 log.cc:1196] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/404c5809014a4d96a933542e6bacf44a
I20260620 23:03:27.514617 20443 ts_tablet_manager.cc:1939] T 404c5809014a4d96a933542e6bacf44a P 6595666887684852bb22ecb873075da7: Deleting consensus metadata
I20260620 23:03:27.514745 20394 tablet_bootstrap.cc:492] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: Bootstrap starting.
I20260620 23:03:27.515028 19877 catalog_manager.cc:4841] TS 6595666887684852bb22ecb873075da7 (127.17.188.65:45589): tablet 404c5809014a4d96a933542e6bacf44a (table test-table [id=571749c12bf04355bcf3b94a7faa0abc]) successfully deleted
I20260620 23:03:27.515156 20394 tablet_bootstrap.cc:654] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:27.515756 20308 tablet_service.cc:1467] Processing CreateTablet for tablet 7af7a6c856d647af952f9d98fa302835 (DEFAULT_TABLE table=test-table [id=bcb1f45fb0c24729bff32ea9db93da52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:27.515775 20394 tablet_bootstrap.cc:492] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: No bootstrap required, opened a new log
I20260620 23:03:27.515817 20394 ts_tablet_manager.cc:1397] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:27.515864 20308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7af7a6c856d647af952f9d98fa302835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:27.515938 20394 raft_consensus.cc:348] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.516001 20394 raft_consensus.cc:374] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:27.516021 20394 raft_consensus.cc:729] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c097f779b534501994a894b85f0e2b0, State: Initialized, Role: FOLLOWER
I20260620 23:03:27.516058 20394 consensus_queue.cc:260] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.516137 20394 ts_tablet_manager.cc:1428] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:27.517207 20395 tablet_bootstrap.cc:492] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: Bootstrap starting.
I20260620 23:03:27.517509 20028 tablet_service.cc:1467] Processing CreateTablet for tablet 7af7a6c856d647af952f9d98fa302835 (DEFAULT_TABLE table=test-table [id=bcb1f45fb0c24729bff32ea9db93da52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:27.517618 20028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7af7a6c856d647af952f9d98fa302835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:27.517709 20395 tablet_bootstrap.cc:654] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:27.518395 20395 tablet_bootstrap.cc:492] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: No bootstrap required, opened a new log
I20260620 23:03:27.518433 20395 ts_tablet_manager.cc:1397] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:27.518564 20395 raft_consensus.cc:348] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.518618 20395 raft_consensus.cc:374] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:27.518638 20395 raft_consensus.cc:729] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97b009ab6e14dfc8f8ba2660cb3f74c, State: Initialized, Role: FOLLOWER
I20260620 23:03:27.518674 20395 consensus_queue.cc:260] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.518718 20396 tablet_bootstrap.cc:492] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: Bootstrap starting.
I20260620 23:03:27.518751 20395 ts_tablet_manager.cc:1428] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:27.519102 20396 tablet_bootstrap.cc:654] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:27.520102 20441 tablet_replica.cc:330] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: stopping tablet replica
I20260620 23:03:27.520227 20441 raft_consensus.cc:2229] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:27.520346 20441 raft_consensus.cc:2258] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:27.520783 20441 ts_tablet_manager.cc:1905] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:27.521425 20442 tablet_replica.cc:330] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: stopping tablet replica
I20260620 23:03:27.521564 20442 raft_consensus.cc:2229] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:27.521659 20442 raft_consensus.cc:2258] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:27.522073 20442 ts_tablet_manager.cc:1905] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:27.522743 20441 ts_tablet_manager.cc:1918] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:03:27.522863 20441 log.cc:1196] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/404c5809014a4d96a933542e6bacf44a
I20260620 23:03:27.523463 20441 ts_tablet_manager.cc:1939] T 404c5809014a4d96a933542e6bacf44a P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting consensus metadata
I20260620 23:03:27.523582 20442 ts_tablet_manager.cc:1918] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:03:27.523697 20442 log.cc:1196] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/404c5809014a4d96a933542e6bacf44a
I20260620 23:03:27.523898 19875 catalog_manager.cc:4841] TS f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293): tablet 404c5809014a4d96a933542e6bacf44a (table test-table [id=571749c12bf04355bcf3b94a7faa0abc]) successfully deleted
I20260620 23:03:27.523998 20442 ts_tablet_manager.cc:1939] T 404c5809014a4d96a933542e6bacf44a P 5c097f779b534501994a894b85f0e2b0: Deleting consensus metadata
I20260620 23:03:27.524428 19878 catalog_manager.cc:4841] TS 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507): tablet 404c5809014a4d96a933542e6bacf44a (table test-table [id=571749c12bf04355bcf3b94a7faa0abc]) successfully deleted
I20260620 23:03:27.525744 20396 tablet_bootstrap.cc:492] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: No bootstrap required, opened a new log
I20260620 23:03:27.525796 20396 ts_tablet_manager.cc:1397] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260620 23:03:27.525923 20396 raft_consensus.cc:348] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.525979 20396 raft_consensus.cc:374] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:27.525998 20396 raft_consensus.cc:729] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6595666887684852bb22ecb873075da7, State: Initialized, Role: FOLLOWER
I20260620 23:03:27.526041 20396 consensus_queue.cc:260] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.526122 20396 ts_tablet_manager.cc:1428] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:27.613670 20400 raft_consensus.cc:482] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:27.613785 20400 raft_consensus.cc:504] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.614262 20400 leader_election.cc:288] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6595666887684852bb22ecb873075da7 (127.17.188.65:45589), f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293)
I20260620 23:03:27.617959 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.614540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:39245 (local address 127.17.188.65:45589)
0620 23:03:27.614586 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.614588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.614785 (+ 197us) server_negotiation.cc:407] Connection header received
0620 23:03:27.614884 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.614885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.614898 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.614950 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:27.615112 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.615564 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.617128 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.617564 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.617650 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.617658 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.617845 (+ 187us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.617888 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:03:27.618049 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.614546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:60775 (local address 127.17.188.67:40293)
0620 23:03:27.614625 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.614627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.614634 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:03:27.614712 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.614713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.614719 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.614795 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:27.614963 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.615680 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.617114 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.617705 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.617783 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.617792 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.617934 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.617980 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 23:03:27.618067 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.614388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:40293 (local address 127.17.188.66:60775)
0620 23:03:27.614553 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.614567 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.614639 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.614801 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.614802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.614804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:27.614880 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.614882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.615762 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.615763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.617015 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.617018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.617786 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.617787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.617859 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.617862 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.617913 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.617955 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":33,"thread_start_us":59,"threads_started":1}
I20260620 23:03:27.618203 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.614388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45589 (local address 127.17.188.66:39245)
0620 23:03:27.614735 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.614743 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.614808 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.614991 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.614996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.614998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:27.615042 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.615043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.615656 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.615657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.617015 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.617018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.617650 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.617651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.617740 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.617761 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.618083 (+ 322us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.618123 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":75,"thread_start_us":45,"threads_started":1}
I20260620 23:03:27.618403 20048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af7a6c856d647af952f9d98fa302835" candidate_uuid: "5c097f779b534501994a894b85f0e2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6595666887684852bb22ecb873075da7" is_pre_election: true
I20260620 23:03:27.618503 20048 raft_consensus.cc:2454] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c097f779b534501994a894b85f0e2b0 in term 0.
I20260620 23:03:27.618645 20328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af7a6c856d647af952f9d98fa302835" candidate_uuid: "5c097f779b534501994a894b85f0e2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" is_pre_election: true
I20260620 23:03:27.618654 20121 leader_election.cc:302] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c097f779b534501994a894b85f0e2b0, 6595666887684852bb22ecb873075da7; no voters:
I20260620 23:03:27.618728 20328 raft_consensus.cc:2454] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c097f779b534501994a894b85f0e2b0 in term 0.
I20260620 23:03:27.618805 20400 raft_consensus.cc:2781] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:27.618839 20400 raft_consensus.cc:482] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:27.618856 20400 raft_consensus.cc:3037] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:27.619652 20400 raft_consensus.cc:504] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.619767 20400 leader_election.cc:288] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [CANDIDATE]: Term 1 election: Requested vote from peers 6595666887684852bb22ecb873075da7 (127.17.188.65:45589), f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293)
I20260620 23:03:27.619946 20048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af7a6c856d647af952f9d98fa302835" candidate_uuid: "5c097f779b534501994a894b85f0e2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6595666887684852bb22ecb873075da7"
I20260620 23:03:27.619967 20328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7af7a6c856d647af952f9d98fa302835" candidate_uuid: "5c097f779b534501994a894b85f0e2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c"
I20260620 23:03:27.620028 20328 raft_consensus.cc:3037] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:27.620031 20048 raft_consensus.cc:3037] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:27.620621 20048 raft_consensus.cc:2454] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c097f779b534501994a894b85f0e2b0 in term 1.
I20260620 23:03:27.620726 20328 raft_consensus.cc:2454] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c097f779b534501994a894b85f0e2b0 in term 1.
I20260620 23:03:27.620792 20121 leader_election.cc:302] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c097f779b534501994a894b85f0e2b0, 6595666887684852bb22ecb873075da7; no voters:
I20260620 23:03:27.620877 20400 raft_consensus.cc:2781] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:27.621021 20400 raft_consensus.cc:686] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 1 LEADER]: Becoming Leader. State: Replica: 5c097f779b534501994a894b85f0e2b0, State: Running, Role: LEADER
I20260620 23:03:27.621079 20400 consensus_queue.cc:237] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:27.621788 19890 catalog_manager.cc:5495] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c097f779b534501994a894b85f0e2b0 (127.17.188.66). New cstate: current_term: 1 leader_uuid: "5c097f779b534501994a894b85f0e2b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:27.640336 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.638350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.17.188.66:46507)
0620 23:03:27.638409 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.638411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.638434 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:03:27.638488 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.638507 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.638523 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.638590 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:27.638666 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.639085 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.639720 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.639881 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.639927 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.640053 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.640138 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.640186 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.640215 (+ 29us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.640231 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:03:27.644979 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.642637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:51963 (local address 127.17.188.65:45589)
0620 23:03:27.642697 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.642700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.642802 (+ 102us) server_negotiation.cc:407] Connection header received
0620 23:03:27.642869 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.642870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.642879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.642966 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:27.643103 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.643568 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.644323 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.644497 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.644562 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.644772 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.644801 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.644845 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.644895 (+ 50us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.644913 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:03:27.645167 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.642549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45589 (local address 127.17.188.66:51963)
0620 23:03:27.642722 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.642734 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.642814 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.642973 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.642974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.642975 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:03:27.643037 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.643038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.643655 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.643656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.644209 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.644210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.644588 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.644589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.644681 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.644693 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.644844 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.644845 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.644891 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.644923 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260620 23:03:27.647128 20048 raft_consensus.cc:1261] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Refusing update from remote peer 5c097f779b534501994a894b85f0e2b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:27.647140 20328 raft_consensus.cc:1261] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Refusing update from remote peer 5c097f779b534501994a894b85f0e2b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:27.647348 20449 consensus_queue.cc:1035] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:27.647480 20400 consensus_queue.cc:1035] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:27.654042 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.651665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:46507 (local address 127.17.188.65:56959)
0620 23:03:27.651719 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:27.651729 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:27.651766 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:27.651953 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:27.651954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:27.651955 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:03:27.651990 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.651991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.652689 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.652689 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.653218 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:27.653219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.653664 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.653664 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:27.653723 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.653729 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.653909 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.653911 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:03:27.653953 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 23:03:27.653995 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 23:03:27.654045 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:27.651817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:56959 (local address 127.17.188.66:46507)
0620 23:03:27.651867 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:27.651869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:27.651876 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:03:27.651894 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:27.651895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:27.651903 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:27.651950 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:27.652035 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.652610 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.653331 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:27.653606 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:27.653666 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:27.653815 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:27.653843 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:27.653886 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:27.653976 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 23:03:27.653995 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260620 23:03:27.654420 20423 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 23:03:29.172163 19890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:29.172269 19890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:29.173043 19890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b: Sending DeleteTablet for 3 replicas of tablet 7af7a6c856d647af952f9d98fa302835
I20260620 23:03:29.173265 20028 tablet_service.cc:1514] Processing DeleteTablet for tablet 7af7a6c856d647af952f9d98fa302835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:29 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:33280
I20260620 23:03:29.173265 20168 tablet_service.cc:1514] Processing DeleteTablet for tablet 7af7a6c856d647af952f9d98fa302835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:29 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:54248
I20260620 23:03:29.173290 20308 tablet_service.cc:1514] Processing DeleteTablet for tablet 7af7a6c856d647af952f9d98fa302835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:29 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56992
I20260620 23:03:29.173482 20478 tablet_replica.cc:330] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: stopping tablet replica
I20260620 23:03:29.173548 20478 raft_consensus.cc:2229] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:29.173595 20478 raft_consensus.cc:2258] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:29.173692 20479 tablet_replica.cc:330] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: stopping tablet replica
I20260620 23:03:29.173873 20478 ts_tablet_manager.cc:1905] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:29.175034 20479 raft_consensus.cc:2229] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:29.175158 20479 raft_consensus.cc:2258] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:29.175288 20478 ts_tablet_manager.cc:1918] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 23:03:29.175364 20478 log.cc:1196] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/7af7a6c856d647af952f9d98fa302835
I20260620 23:03:29.175611 20478 ts_tablet_manager.cc:1939] T 7af7a6c856d647af952f9d98fa302835 P 6595666887684852bb22ecb873075da7: Deleting consensus metadata
I20260620 23:03:29.175926 19877 catalog_manager.cc:4841] TS 6595666887684852bb22ecb873075da7 (127.17.188.65:45589): tablet 7af7a6c856d647af952f9d98fa302835 (table test-table [id=bcb1f45fb0c24729bff32ea9db93da52]) successfully deleted
I20260620 23:03:29.176031 20479 ts_tablet_manager.cc:1905] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:29.176074 20480 tablet_replica.cc:330] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: stopping tablet replica
I20260620 23:03:29.176146 20480 raft_consensus.cc:2229] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:29.176192 20480 raft_consensus.cc:2258] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:29.176429 20480 ts_tablet_manager.cc:1905] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:29.177402 20479 ts_tablet_manager.cc:1918] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 23:03:29.177471 20479 log.cc:1196] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/7af7a6c856d647af952f9d98fa302835
I20260620 23:03:29.177726 20479 ts_tablet_manager.cc:1939] T 7af7a6c856d647af952f9d98fa302835 P 5c097f779b534501994a894b85f0e2b0: Deleting consensus metadata
I20260620 23:03:29.178021 19878 catalog_manager.cc:4841] TS 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507): tablet 7af7a6c856d647af952f9d98fa302835 (table test-table [id=bcb1f45fb0c24729bff32ea9db93da52]) successfully deleted
I20260620 23:03:29.178292 20480 ts_tablet_manager.cc:1918] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 23:03:29.178354 20480 log.cc:1196] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/7af7a6c856d647af952f9d98fa302835
I20260620 23:03:29.178567 20480 ts_tablet_manager.cc:1939] T 7af7a6c856d647af952f9d98fa302835 P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting consensus metadata
I20260620 23:03:29.178895 19875 catalog_manager.cc:4841] TS f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293): tablet 7af7a6c856d647af952f9d98fa302835 (table test-table [id=bcb1f45fb0c24729bff32ea9db93da52]) successfully deleted
Jun 20 23:03:29 dist-test-slave-zpss krb5kdc[19846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996609, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:03:29 dist-test-slave-zpss krb5kdc[19846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996609, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:29.196273 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.189628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48850 (local address 127.17.188.126:45911)
0620 23:03:29.189785 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:29.189787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:29.189815 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:03:29.189900 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:29.189901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:29.189911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:29.189981 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:29.190539 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.191033 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.191711 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.191905 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.191951 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:29.194218 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:29.194219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:29.194219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:29.194225 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:29.195451 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:29.195735 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:29.195735 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:29.195736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:29.195781 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:29.195918 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:29.195918 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:29.195919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:29.196003 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:29.196070 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:29.196160 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 23:03:29.196213 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260620 23:03:29.197777 19890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:29.197937 19890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:29.199219 20168 tablet_service.cc:1467] Processing CreateTablet for tablet 41f0c33672ae48139ffe7f323d751ee8 (DEFAULT_TABLE table=test-table [id=d9220ebad5ad4eb2994120d8c46f0660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:29.199249 20308 tablet_service.cc:1467] Processing CreateTablet for tablet 41f0c33672ae48139ffe7f323d751ee8 (DEFAULT_TABLE table=test-table [id=d9220ebad5ad4eb2994120d8c46f0660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:29.199231 20028 tablet_service.cc:1467] Processing CreateTablet for tablet 41f0c33672ae48139ffe7f323d751ee8 (DEFAULT_TABLE table=test-table [id=d9220ebad5ad4eb2994120d8c46f0660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:29.199344 20168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f0c33672ae48139ffe7f323d751ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:29.199343 20308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f0c33672ae48139ffe7f323d751ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:29.199369 20028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f0c33672ae48139ffe7f323d751ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:29.200428 20489 tablet_bootstrap.cc:492] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: Bootstrap starting.
I20260620 23:03:29.200672 20490 tablet_bootstrap.cc:492] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: Bootstrap starting.
I20260620 23:03:29.200793 20489 tablet_bootstrap.cc:654] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:29.201061 20490 tablet_bootstrap.cc:654] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:29.201377 20491 tablet_bootstrap.cc:492] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: Bootstrap starting.
I20260620 23:03:29.201418 20489 tablet_bootstrap.cc:492] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: No bootstrap required, opened a new log
I20260620 23:03:29.201489 20489 ts_tablet_manager.cc:1397] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:29.201599 20489 raft_consensus.cc:348] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.201642 20490 tablet_bootstrap.cc:492] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: No bootstrap required, opened a new log
I20260620 23:03:29.201684 20489 raft_consensus.cc:374] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:29.201685 20490 ts_tablet_manager.cc:1397] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:29.201711 20491 tablet_bootstrap.cc:654] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:29.201730 20489 raft_consensus.cc:729] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6595666887684852bb22ecb873075da7, State: Initialized, Role: FOLLOWER
I20260620 23:03:29.201787 20489 consensus_queue.cc:260] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.201814 20490 raft_consensus.cc:348] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.201920 20490 raft_consensus.cc:374] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:29.201973 20490 raft_consensus.cc:729] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c097f779b534501994a894b85f0e2b0, State: Initialized, Role: FOLLOWER
I20260620 23:03:29.201999 20489 ts_tablet_manager.cc:1428] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.202035 20490 consensus_queue.cc:260] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.202164 20490 ts_tablet_manager.cc:1428] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.202243 20491 tablet_bootstrap.cc:492] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: No bootstrap required, opened a new log
I20260620 23:03:29.202287 20491 ts_tablet_manager.cc:1397] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:29.202389 20491 raft_consensus.cc:348] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.202456 20491 raft_consensus.cc:374] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:29.202476 20491 raft_consensus.cc:729] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97b009ab6e14dfc8f8ba2660cb3f74c, State: Initialized, Role: FOLLOWER
I20260620 23:03:29.202529 20491 consensus_queue.cc:260] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.202679 20491 ts_tablet_manager.cc:1428] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:29.399510 20495 raft_consensus.cc:482] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:29.399610 20495 raft_consensus.cc:504] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.399773 20495 leader_election.cc:288] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6595666887684852bb22ecb873075da7 (127.17.188.65:45589), 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507)
I20260620 23:03:29.400002 20048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f0c33672ae48139ffe7f323d751ee8" candidate_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6595666887684852bb22ecb873075da7" is_pre_election: true
I20260620 23:03:29.400102 20048 raft_consensus.cc:2454] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f97b009ab6e14dfc8f8ba2660cb3f74c in term 0.
I20260620 23:03:29.400023 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f0c33672ae48139ffe7f323d751ee8" candidate_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c097f779b534501994a894b85f0e2b0" is_pre_election: true
I20260620 23:03:29.400142 20187 raft_consensus.cc:2454] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f97b009ab6e14dfc8f8ba2660cb3f74c in term 0.
I20260620 23:03:29.400272 20261 leader_election.cc:302] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6595666887684852bb22ecb873075da7, f97b009ab6e14dfc8f8ba2660cb3f74c; no voters:
I20260620 23:03:29.400377 20495 raft_consensus.cc:2781] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:29.400416 20495 raft_consensus.cc:482] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:29.400430 20495 raft_consensus.cc:3037] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:29.401124 20495 raft_consensus.cc:504] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.401263 20495 leader_election.cc:288] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [CANDIDATE]: Term 1 election: Requested vote from peers 6595666887684852bb22ecb873075da7 (127.17.188.65:45589), 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507)
I20260620 23:03:29.401510 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f0c33672ae48139ffe7f323d751ee8" candidate_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c097f779b534501994a894b85f0e2b0"
I20260620 23:03:29.401587 20187 raft_consensus.cc:3037] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:29.401566 20048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f0c33672ae48139ffe7f323d751ee8" candidate_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6595666887684852bb22ecb873075da7"
I20260620 23:03:29.401678 20048 raft_consensus.cc:3037] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:29.402254 20048 raft_consensus.cc:2454] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f97b009ab6e14dfc8f8ba2660cb3f74c in term 1.
I20260620 23:03:29.402284 20187 raft_consensus.cc:2454] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f97b009ab6e14dfc8f8ba2660cb3f74c in term 1.
I20260620 23:03:29.402426 20261 leader_election.cc:302] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6595666887684852bb22ecb873075da7, f97b009ab6e14dfc8f8ba2660cb3f74c; no voters:
I20260620 23:03:29.402520 20495 raft_consensus.cc:2781] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:29.402679 20495 raft_consensus.cc:686] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 LEADER]: Becoming Leader. State: Replica: f97b009ab6e14dfc8f8ba2660cb3f74c, State: Running, Role: LEADER
I20260620 23:03:29.402726 20495 consensus_queue.cc:237] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.403324 19891 catalog_manager.cc:5495] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c reported cstate change: term changed from 0 to 1, leader changed from <none> to f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67). New cstate: current_term: 1 leader_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } health_report { overall_health: HEALTHY } } }
I20260620 23:03:29.429342 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.427058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57002 (local address 127.17.188.67:40293)
0620 23:03:29.427209 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:29.427211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:29.427219 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:29.427240 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:29.427240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:29.427258 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:29.427314 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:29.427439 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.427890 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.428600 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.428796 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.428860 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:29.429007 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:29.429033 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:29.429073 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:29.429123 (+ 50us) server_negotiation.cc:299] Negotiation successful
0620 23:03:29.429156 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":108,"threads_started":1}
I20260620 23:03:29.431423 20187 raft_consensus.cc:1261] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Refusing update from remote peer f97b009ab6e14dfc8f8ba2660cb3f74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:29.431470 20048 raft_consensus.cc:1261] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Refusing update from remote peer f97b009ab6e14dfc8f8ba2660cb3f74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:29.431638 20495 consensus_queue.cc:1035] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:29.431761 20496 consensus_queue.cc:1035] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:29.441553 19889 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:29.441627 19889 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:29.442171 19889 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b: Sending DeleteTablet for 3 replicas of tablet 41f0c33672ae48139ffe7f323d751ee8
I20260620 23:03:29.442412 20028 tablet_service.cc:1514] Processing DeleteTablet for tablet 41f0c33672ae48139ffe7f323d751ee8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:29 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:33280
I20260620 23:03:29.442525 20168 tablet_service.cc:1514] Processing DeleteTablet for tablet 41f0c33672ae48139ffe7f323d751ee8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:29 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:54248
I20260620 23:03:29.442652 20478 tablet_replica.cc:330] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: stopping tablet replica
I20260620 23:03:29.442655 20479 tablet_replica.cc:330] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: stopping tablet replica
I20260620 23:03:29.442710 20478 raft_consensus.cc:2229] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:29.442716 20479 raft_consensus.cc:2229] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:29.442673 19889 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 23:03:29.442751 20479 raft_consensus.cc:2258] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:29.442751 20478 raft_consensus.cc:2258] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 23:03:29.442801 19889 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:29.442878 20308 tablet_service.cc:1514] Processing DeleteTablet for tablet 41f0c33672ae48139ffe7f323d751ee8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:29 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56992
I20260620 23:03:29.442970 20480 tablet_replica.cc:330] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: stopping tablet replica
I20260620 23:03:29.443020 20480 raft_consensus.cc:2229] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:29.443073 20480 raft_consensus.cc:2258] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:29.443143 20478 ts_tablet_manager.cc:1905] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:29.443248 20479 ts_tablet_manager.cc:1905] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:29.443995 20168 tablet_service.cc:1467] Processing CreateTablet for tablet 4962cacd1dcb4781a8d6c2f52d990bbd (DEFAULT_TABLE table=test-table [id=693be4c60f2c43cd899628388b03edc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:29.444154 20168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4962cacd1dcb4781a8d6c2f52d990bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:29.444455 20308 tablet_service.cc:1467] Processing CreateTablet for tablet 4962cacd1dcb4781a8d6c2f52d990bbd (DEFAULT_TABLE table=test-table [id=693be4c60f2c43cd899628388b03edc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:29.444561 20308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4962cacd1dcb4781a8d6c2f52d990bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:29.444676 20478 ts_tablet_manager.cc:1918] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:03:29.444720 20478 log.cc:1196] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/41f0c33672ae48139ffe7f323d751ee8
I20260620 23:03:29.444989 20478 ts_tablet_manager.cc:1939] T 41f0c33672ae48139ffe7f323d751ee8 P 6595666887684852bb22ecb873075da7: Deleting consensus metadata
I20260620 23:03:29.445070 20479 ts_tablet_manager.cc:1918] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:03:29.445119 20028 tablet_service.cc:1467] Processing CreateTablet for tablet 4962cacd1dcb4781a8d6c2f52d990bbd (DEFAULT_TABLE table=test-table [id=693be4c60f2c43cd899628388b03edc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:29.445137 20479 log.cc:1196] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/41f0c33672ae48139ffe7f323d751ee8
I20260620 23:03:29.445189 20028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4962cacd1dcb4781a8d6c2f52d990bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:29.445369 20479 ts_tablet_manager.cc:1939] T 41f0c33672ae48139ffe7f323d751ee8 P 5c097f779b534501994a894b85f0e2b0: Deleting consensus metadata
I20260620 23:03:29.445986 20491 tablet_bootstrap.cc:492] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: Bootstrap starting.
I20260620 23:03:29.445993 20490 tablet_bootstrap.cc:492] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: Bootstrap starting.
I20260620 23:03:29.446269 20490 tablet_bootstrap.cc:654] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:29.446386 20491 tablet_bootstrap.cc:654] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:29.446425 20489 tablet_bootstrap.cc:492] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7: Bootstrap starting.
I20260620 23:03:29.446702 19878 catalog_manager.cc:4841] TS 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507): tablet 41f0c33672ae48139ffe7f323d751ee8 (table test-table [id=d9220ebad5ad4eb2994120d8c46f0660]) successfully deleted
I20260620 23:03:29.446768 20489 tablet_bootstrap.cc:654] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:29.446950 20490 tablet_bootstrap.cc:492] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: No bootstrap required, opened a new log
I20260620 23:03:29.447002 20490 ts_tablet_manager.cc:1397] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:29.447141 20490 raft_consensus.cc:348] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.447224 20490 raft_consensus.cc:374] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:29.447252 20490 raft_consensus.cc:729] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c097f779b534501994a894b85f0e2b0, State: Initialized, Role: FOLLOWER
I20260620 23:03:29.447288 19877 catalog_manager.cc:4841] TS 6595666887684852bb22ecb873075da7 (127.17.188.65:45589): tablet 41f0c33672ae48139ffe7f323d751ee8 (table test-table [id=d9220ebad5ad4eb2994120d8c46f0660]) successfully deleted
I20260620 23:03:29.447306 20490 consensus_queue.cc:260] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.447425 20490 ts_tablet_manager.cc:1428] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:29.447436 20489 tablet_bootstrap.cc:492] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7: No bootstrap required, opened a new log
I20260620 23:03:29.447477 20489 ts_tablet_manager.cc:1397] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:29.447500 20480 ts_tablet_manager.cc:1905] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:29.447567 20489 raft_consensus.cc:348] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.447635 20489 raft_consensus.cc:374] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:29.447655 20489 raft_consensus.cc:729] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6595666887684852bb22ecb873075da7, State: Initialized, Role: FOLLOWER
I20260620 23:03:29.447693 20489 consensus_queue.cc:260] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.447763 20489 ts_tablet_manager.cc:1428] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.448729 20480 ts_tablet_manager.cc:1918] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:03:29.448764 20480 log.cc:1196] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/41f0c33672ae48139ffe7f323d751ee8
I20260620 23:03:29.448910 20480 ts_tablet_manager.cc:1939] T 41f0c33672ae48139ffe7f323d751ee8 P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting consensus metadata
I20260620 23:03:29.449100 20491 tablet_bootstrap.cc:492] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: No bootstrap required, opened a new log
I20260620 23:03:29.449141 20491 ts_tablet_manager.cc:1397] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 23:03:29.449255 20491 raft_consensus.cc:348] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.449313 20491 raft_consensus.cc:374] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:29.449309 19875 catalog_manager.cc:4841] TS f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293): tablet 41f0c33672ae48139ffe7f323d751ee8 (table test-table [id=d9220ebad5ad4eb2994120d8c46f0660]) successfully deleted
I20260620 23:03:29.449352 20491 raft_consensus.cc:729] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97b009ab6e14dfc8f8ba2660cb3f74c, State: Initialized, Role: FOLLOWER
I20260620 23:03:29.449397 20491 consensus_queue.cc:260] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.449493 20491 ts_tablet_manager.cc:1428] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.551285 20460 raft_consensus.cc:482] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:29.551365 20460 raft_consensus.cc:504] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.551527 20460 leader_election.cc:288] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507), f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293)
I20260620 23:03:29.551680 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4962cacd1dcb4781a8d6c2f52d990bbd" candidate_uuid: "6595666887684852bb22ecb873075da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c097f779b534501994a894b85f0e2b0" is_pre_election: true
I20260620 23:03:29.551735 20328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4962cacd1dcb4781a8d6c2f52d990bbd" candidate_uuid: "6595666887684852bb22ecb873075da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" is_pre_election: true
I20260620 23:03:29.551798 20187 raft_consensus.cc:2454] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6595666887684852bb22ecb873075da7 in term 0.
I20260620 23:03:29.551808 20328 raft_consensus.cc:2454] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6595666887684852bb22ecb873075da7 in term 0.
I20260620 23:03:29.551951 19982 leader_election.cc:302] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c097f779b534501994a894b85f0e2b0, 6595666887684852bb22ecb873075da7; no voters:
I20260620 23:03:29.552045 20460 raft_consensus.cc:2781] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:29.552084 20460 raft_consensus.cc:482] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:29.552110 20460 raft_consensus.cc:3037] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:29.552786 20460 raft_consensus.cc:504] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.552917 20460 leader_election.cc:288] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [CANDIDATE]: Term 1 election: Requested vote from peers 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507), f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293)
I20260620 23:03:29.553058 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4962cacd1dcb4781a8d6c2f52d990bbd" candidate_uuid: "6595666887684852bb22ecb873075da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c097f779b534501994a894b85f0e2b0"
I20260620 23:03:29.553071 20328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4962cacd1dcb4781a8d6c2f52d990bbd" candidate_uuid: "6595666887684852bb22ecb873075da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c"
I20260620 23:03:29.553151 20328 raft_consensus.cc:3037] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:29.553154 20187 raft_consensus.cc:3037] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:29.553810 20187 raft_consensus.cc:2454] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6595666887684852bb22ecb873075da7 in term 1.
I20260620 23:03:29.553820 20328 raft_consensus.cc:2454] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6595666887684852bb22ecb873075da7 in term 1.
I20260620 23:03:29.554009 19982 leader_election.cc:302] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c097f779b534501994a894b85f0e2b0, 6595666887684852bb22ecb873075da7; no voters:
I20260620 23:03:29.554131 20460 raft_consensus.cc:2781] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:29.554203 20460 raft_consensus.cc:686] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [term 1 LEADER]: Becoming Leader. State: Replica: 6595666887684852bb22ecb873075da7, State: Running, Role: LEADER
I20260620 23:03:29.554250 20460 consensus_queue.cc:237] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } }
I20260620 23:03:29.554791 19891 catalog_manager.cc:5495] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6595666887684852bb22ecb873075da7 (127.17.188.65). New cstate: current_term: 1 leader_uuid: "6595666887684852bb22ecb873075da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6595666887684852bb22ecb873075da7" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:29.568516 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.566190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33296 (local address 127.17.188.65:45589)
0620 23:03:29.566325 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:29.566326 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:29.566332 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:03:29.566352 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:29.566352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:29.566361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:29.566437 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:29.566532 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.567040 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.567788 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.568004 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.568087 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:29.568241 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:29.568274 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:29.568345 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:29.568415 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 23:03:29.568435 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260620 23:03:29.572209 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.570443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:52069 (local address 127.17.188.65:45589)
0620 23:03:29.570509 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:29.570510 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:29.570517 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:03:29.570534 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:29.570535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:29.570542 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:29.570593 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:29.570699 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.571093 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.571716 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.571867 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.571911 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:29.572023 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:29.572042 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:29.572075 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:29.572117 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 23:03:29.572146 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":12,"server-negotiator.queue_time_us":49}
I20260620 23:03:29.572278 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.570284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45589 (local address 127.17.188.65:52069)
0620 23:03:29.570396 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:29.570419 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:29.570467 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:29.570605 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:29.570605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:29.570606 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:03:29.570638 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:29.570638 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.571145 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.571145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:29.571657 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:29.571658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:29.571924 (+ 266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:29.571925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:29.571959 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:29.571962 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:03:29.572086 (+ 124us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:03:29.572087 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:03:29.572117 (+ 30us) client_negotiation.cc:240] Negotiation successful
0620 23:03:29.572157 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260620 23:03:29.573917 20328 raft_consensus.cc:1261] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Refusing update from remote peer 6595666887684852bb22ecb873075da7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:29.573922 20187 raft_consensus.cc:1261] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Refusing update from remote peer 6595666887684852bb22ecb873075da7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:29.574144 20465 consensus_queue.cc:1035] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c097f779b534501994a894b85f0e2b0" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:29.574254 20417 consensus_queue.cc:1035] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f97b009ab6e14dfc8f8ba2660cb3f74c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:03:29.578519 20458 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 23:03:31.095417 19889 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:31.095511 19889 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:31.096232 19889 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7f6c87a25594d9fb75b14dc3024da4b: Sending DeleteTablet for 3 replicas of tablet 4962cacd1dcb4781a8d6c2f52d990bbd
I20260620 23:03:31.096457 20026 tablet_service.cc:1514] Processing DeleteTablet for tablet 4962cacd1dcb4781a8d6c2f52d990bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:31 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:33280
I20260620 23:03:31.096482 20168 tablet_service.cc:1514] Processing DeleteTablet for tablet 4962cacd1dcb4781a8d6c2f52d990bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:31 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:54248
I20260620 23:03:31.096503 20308 tablet_service.cc:1514] Processing DeleteTablet for tablet 4962cacd1dcb4781a8d6c2f52d990bbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:31 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56992
I20260620 23:03:31.096644 20535 tablet_replica.cc:330] T 4962cacd1dcb4781a8d6c2f52d990bbd P 6595666887684852bb22ecb873075da7: stopping tablet replica
I20260620 23:03:31.096647 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 19963
I20260620 23:03:31.098098 20534 tablet_replica.cc:330] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: stopping tablet replica
I20260620 23:03:31.098177 20534 raft_consensus.cc:2229] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:31.098210 20534 raft_consensus.cc:2258] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:31.098321 20536 tablet_replica.cc:330] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: stopping tablet replica
I20260620 23:03:31.098397 20536 raft_consensus.cc:2229] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:31.098459 20536 raft_consensus.cc:2258] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:31.098479 20534 ts_tablet_manager.cc:1905] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:31.098697 20536 ts_tablet_manager.cc:1905] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:31.100224 20536 ts_tablet_manager.cc:1918] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 23:03:31.100296 20536 log.cc:1196] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/4962cacd1dcb4781a8d6c2f52d990bbd
I20260620 23:03:31.100411 20534 ts_tablet_manager.cc:1918] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 23:03:31.100477 20534 log.cc:1196] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/4962cacd1dcb4781a8d6c2f52d990bbd
I20260620 23:03:31.100565 20536 ts_tablet_manager.cc:1939] T 4962cacd1dcb4781a8d6c2f52d990bbd P f97b009ab6e14dfc8f8ba2660cb3f74c: Deleting consensus metadata
I20260620 23:03:31.100751 20534 ts_tablet_manager.cc:1939] T 4962cacd1dcb4781a8d6c2f52d990bbd P 5c097f779b534501994a894b85f0e2b0: Deleting consensus metadata
I20260620 23:03:31.100900 19875 catalog_manager.cc:4841] TS f97b009ab6e14dfc8f8ba2660cb3f74c (127.17.188.67:40293): tablet 4962cacd1dcb4781a8d6c2f52d990bbd (table test-table [id=693be4c60f2c43cd899628388b03edc7]) successfully deleted
I20260620 23:03:31.101046 19878 catalog_manager.cc:4841] TS 5c097f779b534501994a894b85f0e2b0 (127.17.188.66:46507): tablet 4962cacd1dcb4781a8d6c2f52d990bbd (table test-table [id=693be4c60f2c43cd899628388b03edc7]) successfully deleted
W20260620 23:03:31.103574 19877 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:45589 (error 108)
I20260620 23:03:31.103778 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 20103
I20260620 23:03:31.109478 20538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:31.103941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45589 (local address 127.0.0.1:33298)
0620 23:03:31.109383 (+ 5442us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:31.109433 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:45589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5413,"thread_start_us":57,"threads_started":1}
I20260620 23:03:31.109616 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 20243
W20260620 23:03:31.109606 19877 catalog_manager.cc:4584] TS 6595666887684852bb22ecb873075da7 (127.17.188.65:45589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4962cacd1dcb4781a8d6c2f52d990bbd: Network error: Client connection negotiation failed: client connection to 127.17.188.65:45589: connect: Connection refused (error 111)
I20260620 23:03:31.114423 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 19859
2026-06-20T23:03:31Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5748 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:31 dist-test-slave-zpss krb5kdc[20541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:31 dist-test-slave-zpss krb5kdc[20541](info): set up 2 sockets
Jun 20 23:03:31 dist-test-slave-zpss krb5kdc[20541](info): commencing operation
krb5kdc: starting...
W20260620 23:03:32.128986 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[20541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:32.237918 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:38743
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:40623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:38743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:32.316252 20554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:32.316442 20554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:32.316463 20554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:32.317602 20554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:32.317646 20554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:32.317659 20554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:32.317734 20554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:32.317764 20554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:32.319197 20554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:40623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:38743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:38743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:32.319435 20554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:32.319659 20554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:03:32.324740 20554 server_base.cc:1034] running on GCE node
W20260620 23:03:32.324872 20560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:32.324872 20562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:32.324931 20559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:32.325309 20554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:32.325652 20554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:32.326813 20554 hybrid_clock.cc:648] HybridClock initialized: now 1781996612326811 us; error 35 us; skew 500 ppm
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[20541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:32.329145 20554 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:32.329617 20554 webserver.cc:460] Webserver started at http://127.17.188.126:45043/ using document root <none> and password file <none>
I20260620 23:03:32.329838 20554 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:32.329907 20554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:32.330050 20554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:32.331045 20554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "b16955844ecb4a6db14aa81beff45045"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "a6a0d71a04cab2625cab22f9ee6f39b3"
server_key_iv: "aadb892fe6d2cc3b024fe2ba147b9a17"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.331454 20554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "b16955844ecb4a6db14aa81beff45045"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "a6a0d71a04cab2625cab22f9ee6f39b3"
server_key_iv: "aadb892fe6d2cc3b024fe2ba147b9a17"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.332711 20554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:32.333470 20569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.333643 20554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:32.333740 20554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "b16955844ecb4a6db14aa81beff45045"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "a6a0d71a04cab2625cab22f9ee6f39b3"
server_key_iv: "aadb892fe6d2cc3b024fe2ba147b9a17"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.333819 20554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:32.344673 20554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:32.346459 20554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:32.346630 20554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:32.350677 20554 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:38743
I20260620 23:03:32.350718 20621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:38743 every 8 connection(s)
I20260620 23:03:32.351052 20554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:32.351898 20622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:32.351929 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 20554
I20260620 23:03:32.352003 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:32.352164 18161 external_mini_cluster.cc:1428] Setting key 8c8afd302ee09848768108d3c4451399
I20260620 23:03:32.354934 20622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045: Bootstrap starting.
I20260620 23:03:32.355602 20622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:32.355893 20622 log.cc:824] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:32.356545 20622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045: No bootstrap required, opened a new log
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[20541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:32.359048 20622 raft_consensus.cc:348] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16955844ecb4a6db14aa81beff45045" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38743 } }
I20260620 23:03:32.359156 20622 raft_consensus.cc:374] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:32.359200 20622 raft_consensus.cc:729] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16955844ecb4a6db14aa81beff45045, State: Initialized, Role: FOLLOWER
I20260620 23:03:32.359324 20622 consensus_queue.cc:260] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16955844ecb4a6db14aa81beff45045" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38743 } }
I20260620 23:03:32.359387 20622 raft_consensus.cc:388] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:32.359406 20622 raft_consensus.cc:482] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:32.359490 20622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:32.359957 20622 raft_consensus.cc:504] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16955844ecb4a6db14aa81beff45045" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38743 } }
I20260620 23:03:32.360067 20622 leader_election.cc:302] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b16955844ecb4a6db14aa81beff45045; no voters:
I20260620 23:03:32.360248 20622 leader_election.cc:288] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:32.360276 20627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:32.360375 20627 raft_consensus.cc:686] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [term 1 LEADER]: Becoming Leader. State: Replica: b16955844ecb4a6db14aa81beff45045, State: Running, Role: LEADER
I20260620 23:03:32.360450 20627 consensus_queue.cc:237] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16955844ecb4a6db14aa81beff45045" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38743 } }
I20260620 23:03:32.360488 20622 sys_catalog.cc:564] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:32.360762 20627 sys_catalog.cc:455] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b16955844ecb4a6db14aa81beff45045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16955844ecb4a6db14aa81beff45045" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38743 } } }
I20260620 23:03:32.360913 20627 sys_catalog.cc:458] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:32.361093 20627 sys_catalog.cc:455] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b16955844ecb4a6db14aa81beff45045. Latest consensus state: current_term: 1 leader_uuid: "b16955844ecb4a6db14aa81beff45045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16955844ecb4a6db14aa81beff45045" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38743 } } }
I20260620 23:03:32.361163 20627 sys_catalog.cc:458] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:32.361457 20636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:32.362015 20636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:32.362921 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.352663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.17.188.126:38743)
0620 23:03:32.352861 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:32.352869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:32.352898 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:03:32.353370 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:32.353378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:32.353435 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:32.353542 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:32.354325 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.355203 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.356224 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.356684 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.356770 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:32.359166 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:32.359170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:32.359170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:32.359182 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:32.361138 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:32.361583 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:32.361587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:32.361587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:32.361704 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:32.361958 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:32.361959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:32.361960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:32.362263 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:32.362391 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:32.362632 (+ 241us) server_negotiation.cc:299] Negotiation successful
0620 23:03:32.362775 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260620 23:03:32.363651 20636 catalog_manager.cc:1347] Generated new cluster ID: c789ba6f5c8c4ea3873969b05bf10f8f
I20260620 23:03:32.363696 20636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:32.378348 20636 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:32.378826 20636 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:32.384884 20636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b16955844ecb4a6db14aa81beff45045: Generated new TSK 0
I20260620 23:03:32.385038 20636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:03:32.415634 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38743
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:40623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:32.530592 20648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:32.530768 20648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:32.530792 20648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:32.531862 20648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:32.531900 20648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:32.531950 20648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:32.533285 20648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:40623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:32.533578 20648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:32.533828 20648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:32.538375 20653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:32.538471 20654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:32.538383 20656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:32.538700 20648 server_base.cc:1034] running on GCE node
I20260620 23:03:32.538928 20648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:32.539193 20648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:32.540352 20648 hybrid_clock.cc:648] HybridClock initialized: now 1781996612540332 us; error 34 us; skew 500 ppm
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[20541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:32.542768 20648 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:03:32.543175 20648 webserver.cc:460] Webserver started at http://127.17.188.65:37529/ using document root <none> and password file <none>
I20260620 23:03:32.543401 20648 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:32.543467 20648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:32.543610 20648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:32.544611 20648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "f3690e601b2840dda6e50bf39f3118f5"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "a3f8f1447e4bc695cd24431f880c349f"
server_key_iv: "74c8113b84135534c169b13b3f196869"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.544956 20648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "f3690e601b2840dda6e50bf39f3118f5"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "a3f8f1447e4bc695cd24431f880c349f"
server_key_iv: "74c8113b84135534c169b13b3f196869"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.546283 20648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:32.546947 20663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.547108 20648 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:32.547206 20648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "f3690e601b2840dda6e50bf39f3118f5"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "a3f8f1447e4bc695cd24431f880c349f"
server_key_iv: "74c8113b84135534c169b13b3f196869"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.547286 20648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:32.564241 20648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:32.566340 20648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:32.566489 20648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:32.566711 20648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:32.567008 20648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:32.567065 20648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.567103 20648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:32.567130 20648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.572526 20648 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:36895
I20260620 23:03:32.572599 20776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:36895 every 8 connection(s)
I20260620 23:03:32.572921 20648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[20541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:32.580605 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 20648
I20260620 23:03:32.580686 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:32.580883 18161 external_mini_cluster.cc:1428] Setting key 89d2db6e5461ecbfe70e6935a2261eb5
I20260620 23:03:32.581044 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.573641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:58621 (local address 127.17.188.126:38743)
0620 23:03:32.573706 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:32.573708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:32.573903 (+ 195us) server_negotiation.cc:407] Connection header received
0620 23:03:32.574340 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:32.574341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:32.574349 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:32.574399 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:32.575201 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.575654 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.576539 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.576719 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.576767 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:32.578667 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:32.578669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:32.578669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:32.578675 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:32.579767 (+ 1092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:32.580077 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:32.580078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:32.580078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:32.580120 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:32.580325 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:32.580326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:32.580326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:32.580518 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:32.580604 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:32.580874 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 23:03:32.580953 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:03:32.581175 20779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.573629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:38743 (local address 127.17.188.65:58621)
0620 23:03:32.573776 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:32.573814 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:32.574265 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:32.574446 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:32.574449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:32.574974 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:32.575132 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:32.575136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.575743 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.575744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:32.576471 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:32.576472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.576786 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.576787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:32.576876 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:32.577249 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:32.577255 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:32.578591 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:32.579828 (+ 1237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:32.579834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:32.579835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:32.580020 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:32.580189 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:32.580189 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:32.580190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:32.580276 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:32.580597 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:32.580601 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:32.580735 (+ 134us) client_negotiation.cc:769] Sending connection context
0620 23:03:32.580828 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 23:03:32.580963 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260620 23:03:32.581681 20777 heartbeater.cc:347] Connected to a master server at 127.17.188.126:38743
I20260620 23:03:32.581781 20777 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:32.581951 20777 heartbeater.cc:511] Master 127.17.188.126:38743 requested a full tablet report, sending...
I20260620 23:03:32.582434 20585 ts_manager.cc:194] Registered new tserver with Master: f3690e601b2840dda6e50bf39f3118f5 (127.17.188.65:36895)
I20260620 23:03:32.583003 20585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:58621
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:03:32.603379 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38743
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:40623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:32.682799 20782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:32.682972 20782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:32.682996 20782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:32.684218 20782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:32.684265 20782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:32.684290 20782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:32.685670 20782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:40623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:32.685892 20782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:32.686111 20782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:32.690878 20787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:32.690915 20782 server_base.cc:1034] running on GCE node
W20260620 23:03:32.690938 20788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:32.690881 20790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:32.691246 20782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:32.691525 20782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:32.692687 20782 hybrid_clock.cc:648] HybridClock initialized: now 1781996612692669 us; error 30 us; skew 500 ppm
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[20541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:32.695093 20782 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:03:32.695542 20782 webserver.cc:460] Webserver started at http://127.17.188.66:41433/ using document root <none> and password file <none>
I20260620 23:03:32.695762 20782 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:32.695844 20782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:32.695987 20782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:32.696920 20782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "14a72e0334cf4aca9edbe4a982d7549b"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "42c9e566d7b0ccb9ef383dde776c2126"
server_key_iv: "4bf6849537edd811e4ca8f861b6e07be"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.697271 20782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "14a72e0334cf4aca9edbe4a982d7549b"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "42c9e566d7b0ccb9ef383dde776c2126"
server_key_iv: "4bf6849537edd811e4ca8f861b6e07be"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.699050 20782 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260620 23:03:32.699671 20797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.699862 20782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:32.699952 20782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "14a72e0334cf4aca9edbe4a982d7549b"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "42c9e566d7b0ccb9ef383dde776c2126"
server_key_iv: "4bf6849537edd811e4ca8f861b6e07be"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.700033 20782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:32.714818 20782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:32.716432 20782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:32.716585 20782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:32.716796 20782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:32.717104 20782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:32.717157 20782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.717191 20782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:32.717218 20782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.722944 20782 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:44463
I20260620 23:03:32.722976 20910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:44463 every 8 connection(s)
I20260620 23:03:32.723265 20782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:32.727604 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 20782
I20260620 23:03:32.727705 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:32.727885 18161 external_mini_cluster.cc:1428] Setting key 68e3cf4cfd9ae693c51217f45d460b0c
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[20541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:32.732213 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.723910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:38743 (local address 127.17.188.66:36257)
0620 23:03:32.724075 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:32.724107 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:32.724493 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:32.724667 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:32.724670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:32.725171 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:32.725390 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:32.725396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.726043 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.726044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:32.726789 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:32.726791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.727094 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.727095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:32.727182 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:32.727680 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:32.727689 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:32.729255 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:32.730875 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:32.730882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:32.730884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:32.731095 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:32.731309 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:32.731310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:32.731310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:32.731420 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:32.731685 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:32.731692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:32.731821 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 23:03:32.731950 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 23:03:32.732072 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260620 23:03:32.732204 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.723944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:36257 (local address 127.17.188.126:38743)
0620 23:03:32.724015 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:32.724017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:32.724179 (+ 162us) server_negotiation.cc:407] Connection header received
0620 23:03:32.724562 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:32.724563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:32.724572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:32.724620 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:32.725474 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.725963 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.726856 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.727029 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.727078 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:32.729373 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:32.729374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:32.729374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:32.729381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:32.730775 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:32.731172 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:32.731173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:32.731174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:32.731232 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:32.731497 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:32.731498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:32.731498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:32.731615 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:32.731682 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:32.732006 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 23:03:32.732094 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:03:32.732792 20911 heartbeater.cc:347] Connected to a master server at 127.17.188.126:38743
I20260620 23:03:32.732887 20911 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:32.733079 20911 heartbeater.cc:511] Master 127.17.188.126:38743 requested a full tablet report, sending...
I20260620 23:03:32.733457 20585 ts_manager.cc:194] Registered new tserver with Master: 14a72e0334cf4aca9edbe4a982d7549b (127.17.188.66:44463)
I20260620 23:03:32.733870 20585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:36257
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:03:32.750070 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38743
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:40623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:32.827289 20916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:32.827513 20916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:32.827551 20916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:32.828713 20916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:32.828785 20916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:32.828830 20916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:32.830317 20916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:40623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:32.830570 20916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:32.830792 20916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:32.835675 20922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:32.835726 20916 server_base.cc:1034] running on GCE node
W20260620 23:03:32.835647 20924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:32.835644 20921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:32.836052 20916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:32.836274 20916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:32.837387 20916 hybrid_clock.cc:648] HybridClock initialized: now 1781996612837367 us; error 30 us; skew 500 ppm
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[20541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:32.839673 20916 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:03:32.840123 20916 webserver.cc:460] Webserver started at http://127.17.188.67:44825/ using document root <none> and password file <none>
I20260620 23:03:32.840337 20916 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:32.840402 20916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:32.840548 20916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:32.841660 20916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "6eb0b9f1133d4931bbc514eaabf32ad5"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "50ac0dd0d1c07e1b49d50ff9aec444a2"
server_key_iv: "eea46a79a84c7a2a3f1bc2f9e9eb5a94"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.842022 20916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "6eb0b9f1133d4931bbc514eaabf32ad5"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "50ac0dd0d1c07e1b49d50ff9aec444a2"
server_key_iv: "eea46a79a84c7a2a3f1bc2f9e9eb5a94"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.843312 20916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:03:32.844018 20931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.844194 20916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:32.844293 20916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "6eb0b9f1133d4931bbc514eaabf32ad5"
format_stamp: "Formatted at 2026-06-20 23:03:32 on dist-test-slave-zpss"
server_key: "50ac0dd0d1c07e1b49d50ff9aec444a2"
server_key_iv: "eea46a79a84c7a2a3f1bc2f9e9eb5a94"
server_key_version: "encryptionkey@0"
I20260620 23:03:32.844372 20916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:32.862397 20916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:32.864583 20916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:32.864740 20916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:32.864959 20916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:32.865248 20916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:32.865280 20916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.865345 20916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:32.865375 20916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:32.870805 20916 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:39137
I20260620 23:03:32.870872 21044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:39137 every 8 connection(s)
I20260620 23:03:32.871181 20916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:32.874095 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 20916
I20260620 23:03:32.874202 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:32.874394 18161 external_mini_cluster.cc:1428] Setting key 7a8627fafbea543163ff25d384ee6e88
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[20541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:32.879563 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.871886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:46185 (local address 127.17.188.126:38743)
0620 23:03:32.871949 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:32.871951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:32.872155 (+ 204us) server_negotiation.cc:407] Connection header received
0620 23:03:32.872612 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:32.872613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:32.872622 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:32.872677 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:32.873612 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.874091 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.875241 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.875422 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.875474 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:32.877349 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:32.877351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:32.877351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:32.877359 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:32.878493 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:32.878848 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:32.878849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:32.878849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:32.878897 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:32.879095 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:32.879095 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:32.879096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:32.879186 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:32.879234 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:32.879445 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 23:03:32.879502 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:03:32.879760 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.871861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:38743 (local address 127.17.188.67:46185)
0620 23:03:32.872031 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:32.872073 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:32.872533 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:32.872725 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:32.872729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:32.873284 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:32.873536 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:32.873541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.874183 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.874185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:32.875141 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:32.875144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:32.875490 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:32.875491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:32.875585 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:32.875989 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:32.875996 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:32.877235 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:32.878561 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:32.878566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:32.878568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:32.878784 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:32.878953 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:32.878953 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:32.878954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:32.879043 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:32.879238 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:32.879241 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:32.879340 (+ 99us) client_negotiation.cc:769] Sending connection context
0620 23:03:32.879417 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 23:03:32.879523 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260620 23:03:32.880215 21045 heartbeater.cc:347] Connected to a master server at 127.17.188.126:38743
I20260620 23:03:32.880297 21045 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:32.880424 21045 heartbeater.cc:511] Master 127.17.188.126:38743 requested a full tablet report, sending...
I20260620 23:03:32.880726 20585 ts_manager.cc:194] Registered new tserver with Master: 6eb0b9f1133d4931bbc514eaabf32ad5 (127.17.188.67:39137)
I20260620 23:03:32.881093 20585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:46185
I20260620 23:03:32.885938 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 23:03:32.896194 20625 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:03:32.895416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54906 (local address 127.17.188.126:38743)
0620 23:03:32.895473 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:32.895475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:32.895547 (+ 72us) server_negotiation.cc:407] Connection header received
0620 23:03:32.895583 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:32.895584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:32.895591 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:32.895641 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:32.896113 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54906: BlockingRecv error: recv got EOF from 127.0.0.1:54906 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:03:32.897168 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 20648
I20260620 23:03:32.901566 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 20782
I20260620 23:03:32.905272 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 20916
I20260620 23:03:32.909299 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 20554
2026-06-20T23:03:32Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1794 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[21057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[21057](info): set up 2 sockets
Jun 20 23:03:32 dist-test-slave-zpss krb5kdc[21057](info): commencing operation
krb5kdc: starting...
W20260620 23:03:33.944921 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:34 dist-test-slave-zpss krb5kdc[21057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:34.072866 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:44765
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:44923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:44765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:34.148764 21070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:34.148955 21070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:34.148976 21070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:34.150107 21070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:34.150143 21070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:34.150157 21070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:34.150205 21070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:34.150233 21070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:34.151608 21070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:44923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:44765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:44765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:34.151831 21070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:34.152042 21070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:34.156607 21076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:34.156639 21078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:34.156698 21075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:34.156814 21070 server_base.cc:1034] running on GCE node
I20260620 23:03:34.157018 21070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:34.157284 21070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:34.158466 21070 hybrid_clock.cc:648] HybridClock initialized: now 1781996614158454 us; error 28 us; skew 500 ppm
Jun 20 23:03:34 dist-test-slave-zpss krb5kdc[21057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996614, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:34.160826 21070 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:34.161180 21070 webserver.cc:460] Webserver started at http://127.17.188.126:37323/ using document root <none> and password file <none>
I20260620 23:03:34.161384 21070 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:34.161432 21070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:34.161511 21070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:34.162402 21070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "67452672b581475e99235af017bde5c2"
format_stamp: "Formatted at 2026-06-20 23:03:34 on dist-test-slave-zpss"
server_key: "d7fe81781ca8daeb4c108e317d8b666b"
server_key_iv: "ebea364711ee3e6edff2cdefcae45b7b"
server_key_version: "encryptionkey@0"
I20260620 23:03:34.162696 21070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "67452672b581475e99235af017bde5c2"
format_stamp: "Formatted at 2026-06-20 23:03:34 on dist-test-slave-zpss"
server_key: "d7fe81781ca8daeb4c108e317d8b666b"
server_key_iv: "ebea364711ee3e6edff2cdefcae45b7b"
server_key_version: "encryptionkey@0"
I20260620 23:03:34.163902 21070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 23:03:34.164544 21085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:34.164705 21070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:34.164798 21070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "67452672b581475e99235af017bde5c2"
format_stamp: "Formatted at 2026-06-20 23:03:34 on dist-test-slave-zpss"
server_key: "d7fe81781ca8daeb4c108e317d8b666b"
server_key_iv: "ebea364711ee3e6edff2cdefcae45b7b"
server_key_version: "encryptionkey@0"
I20260620 23:03:34.164871 21070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:34.179325 21070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:34.181599 21070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:34.181754 21070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:34.185446 21070 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:44765
I20260620 23:03:34.185485 21137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:44765 every 8 connection(s)
I20260620 23:03:34.185943 21070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:34.186535 21138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:34.186785 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 21070
I20260620 23:03:34.186890 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:34.187075 18161 external_mini_cluster.cc:1428] Setting key fdd4ab523682f0c1663aa41b57a14c41
I20260620 23:03:34.189460 21138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2: Bootstrap starting.
I20260620 23:03:34.190042 21138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:34.190320 21138 log.cc:824] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:34.190958 21138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2: No bootstrap required, opened a new log
I20260620 23:03:34.192670 21138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67452672b581475e99235af017bde5c2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44765 } }
I20260620 23:03:34.192816 21138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:34.192870 21138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67452672b581475e99235af017bde5c2, State: Initialized, Role: FOLLOWER
I20260620 23:03:34.193035 21138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67452672b581475e99235af017bde5c2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44765 } }
I20260620 23:03:34.193147 21138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:34.193194 21138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:34.193241 21138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 23:03:34 dist-test-slave-zpss krb5kdc[21057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:34.193822 21138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67452672b581475e99235af017bde5c2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44765 } }
I20260620 23:03:34.193934 21138 leader_election.cc:302] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67452672b581475e99235af017bde5c2; no voters:
I20260620 23:03:34.194101 21138 leader_election.cc:288] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:34.194154 21143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:34.194320 21143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [term 1 LEADER]: Becoming Leader. State: Replica: 67452672b581475e99235af017bde5c2, State: Running, Role: LEADER
I20260620 23:03:34.194401 21138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:34.194416 21143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67452672b581475e99235af017bde5c2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44765 } }
I20260620 23:03:34.194787 21143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67452672b581475e99235af017bde5c2. Latest consensus state: current_term: 1 leader_uuid: "67452672b581475e99235af017bde5c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67452672b581475e99235af017bde5c2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44765 } } }
I20260620 23:03:34.194885 21143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:34.194795 21145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67452672b581475e99235af017bde5c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67452672b581475e99235af017bde5c2" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44765 } } }
I20260620 23:03:34.195046 21145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:34.195541 21150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:34.196125 21150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:34.197638 21150 catalog_manager.cc:1347] Generated new cluster ID: 60521af085884b78a7e1f36a9ae3c3f6
I20260620 23:03:34.197687 21150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:34.198493 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:34.187574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60614 (local address 127.17.188.126:44765)
0620 23:03:34.187732 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:34.187742 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:34.187771 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:03:34.188174 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:34.188180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:34.188222 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:34.188334 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:34.189111 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:34.190057 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:34.190845 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:34.191310 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:34.191428 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:34.193874 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:34.193879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:34.193879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:34.193891 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:34.196738 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:34.197170 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:34.197173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:34.197173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:34.197292 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:34.197601 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:34.197602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:34.197602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:34.197877 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:34.198031 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:34.198157 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 23:03:34.198293 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260620 23:03:34.199357 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:03:34.219144 21150 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:34.219803 21150 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:34.222414 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:34.218380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.17.188.126:44765)
0620 23:03:34.218438 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:34.218440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:34.218611 (+ 171us) server_negotiation.cc:407] Connection header received
0620 23:03:34.218659 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:34.218660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:34.218671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:34.218731 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:34.219211 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:34.219896 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:34.220883 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:34.221170 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:34.221229 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:34.221963 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:34.221964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:34.221964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:34.222158 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:34.222173 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:34.222179 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:34.222229 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:34.222309 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:03:34.222344 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:03:34.227914 21150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67452672b581475e99235af017bde5c2: Generated new TSK 0
I20260620 23:03:34.228094 21150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:03:34.232661 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 21070
2026-06-20T23:03:34Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1321 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:34 dist-test-slave-zpss krb5kdc[21171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:34 dist-test-slave-zpss krb5kdc[21171](info): set up 2 sockets
Jun 20 23:03:34 dist-test-slave-zpss krb5kdc[21171](info): commencing operation
krb5kdc: starting...
W20260620 23:03:35.253019 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:35.361583 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35223
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:35223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:35.437927 21184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:35.438143 21184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:35.438177 21184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:35.439296 21184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:35.439357 21184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:35.439383 21184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:35.439414 21184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:35.439450 21184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:35.440800 21184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:35223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:35.441059 21184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:35.441282 21184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:03:35.445837 21184 server_base.cc:1034] running on GCE node
W20260620 23:03:35.445819 21189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:35.445798 21192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:35.445812 21190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:35.446285 21184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:35.446502 21184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:35.447638 21184 hybrid_clock.cc:648] HybridClock initialized: now 1781996615447637 us; error 36 us; skew 500 ppm
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:35.450033 21184 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:35.450459 21184 webserver.cc:460] Webserver started at http://127.17.188.126:35805/ using document root <none> and password file <none>
I20260620 23:03:35.450690 21184 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:35.450754 21184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:35.450901 21184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:35.451906 21184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "660d05cd88f741428d20fc2eb3f3d5cf"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "c7f245cfe20d4493281e6dfc255cb4c2"
server_key_iv: "a7e521c56bfd0b9ec57c0af248153d99"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.452250 21184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "660d05cd88f741428d20fc2eb3f3d5cf"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "c7f245cfe20d4493281e6dfc255cb4c2"
server_key_iv: "a7e521c56bfd0b9ec57c0af248153d99"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.453585 21184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:35.454618 21199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.454819 21184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:35.454912 21184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "660d05cd88f741428d20fc2eb3f3d5cf"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "c7f245cfe20d4493281e6dfc255cb4c2"
server_key_iv: "a7e521c56bfd0b9ec57c0af248153d99"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.454994 21184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:35.472735 21184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:35.475081 21184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:35.475244 21184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:35.479393 21184 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:35223
I20260620 23:03:35.479422 21251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:35223 every 8 connection(s)
I20260620 23:03:35.479753 21184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:35.480651 21252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:35.482841 21252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf: Bootstrap starting.
I20260620 23:03:35.483397 21252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:35.483685 21252 log.cc:824] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:35.484412 21252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf: No bootstrap required, opened a new log
I20260620 23:03:35.485670 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 21184
I20260620 23:03:35.485771 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:35.485935 18161 external_mini_cluster.cc:1428] Setting key edd86fe5c8276eb9023447d60f769ee8
I20260620 23:03:35.486251 21252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "660d05cd88f741428d20fc2eb3f3d5cf" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35223 } }
I20260620 23:03:35.486449 21252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:35.486534 21252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 660d05cd88f741428d20fc2eb3f3d5cf, State: Initialized, Role: FOLLOWER
I20260620 23:03:35.486681 21252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "660d05cd88f741428d20fc2eb3f3d5cf" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35223 } }
I20260620 23:03:35.486761 21252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:35.486784 21252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:35.486850 21252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:35.487959 21252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "660d05cd88f741428d20fc2eb3f3d5cf" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35223 } }
I20260620 23:03:35.488045 21252 leader_election.cc:302] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 660d05cd88f741428d20fc2eb3f3d5cf; no voters:
I20260620 23:03:35.488216 21252 leader_election.cc:288] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:35.488240 21257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:35.488348 21257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [term 1 LEADER]: Becoming Leader. State: Replica: 660d05cd88f741428d20fc2eb3f3d5cf, State: Running, Role: LEADER
I20260620 23:03:35.488471 21252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:35.488484 21257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "660d05cd88f741428d20fc2eb3f3d5cf" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35223 } }
I20260620 23:03:35.488965 21259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 660d05cd88f741428d20fc2eb3f3d5cf. Latest consensus state: current_term: 1 leader_uuid: "660d05cd88f741428d20fc2eb3f3d5cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "660d05cd88f741428d20fc2eb3f3d5cf" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35223 } } }
I20260620 23:03:35.489056 21259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:35.489259 21258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "660d05cd88f741428d20fc2eb3f3d5cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "660d05cd88f741428d20fc2eb3f3d5cf" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35223 } } }
I20260620 23:03:35.489591 21258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:35.489732 21265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:35.490267 21265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:35.491788 21265 catalog_manager.cc:1347] Generated new cluster ID: 623c28e039a7467f8d6f9852b30461ba
I20260620 23:03:35.491842 21265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:35.497301 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.486423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.17.188.126:35223)
0620 23:03:35.486629 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:35.486638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:35.486672 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:03:35.486793 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:35.486800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:35.486852 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:35.487026 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:35.487797 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.488925 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.490102 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.490689 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.490810 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:35.493479 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:35.493483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:35.493484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:35.493494 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:35.495351 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:35.495774 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:35.495777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:35.495777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:35.495887 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:35.496121 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:35.496122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:35.496122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:35.496395 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:35.496890 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:35.496973 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 23:03:35.497111 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260620 23:03:35.528476 21265 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:35.528997 21265 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:35.534622 21265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 660d05cd88f741428d20fc2eb3f3d5cf: Generated new TSK 0
I20260620 23:03:35.534808 21265 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:03:35.560369 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35223
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:35.636251 21278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:35.636421 21278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:35.636441 21278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:35.637506 21278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:35.637543 21278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:35.637567 21278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:35.638934 21278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:35.639144 21278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:35.639356 21278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:35.643977 21284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:35.643983 21286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:35.644027 21278 server_base.cc:1034] running on GCE node
W20260620 23:03:35.643996 21283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:35.644349 21278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:35.644593 21278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:35.645757 21278 hybrid_clock.cc:648] HybridClock initialized: now 1781996615645744 us; error 25 us; skew 500 ppm
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:35.648097 21278 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:03:35.648502 21278 webserver.cc:460] Webserver started at http://127.17.188.65:40861/ using document root <none> and password file <none>
I20260620 23:03:35.648725 21278 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:35.648793 21278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:35.648940 21278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:35.649936 21278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "51364830c0c4420eb3e075f77b619533"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "85b5064122aa649be9d7a702d4f78e54"
server_key_iv: "375eecd19844b08dd44332fd02b040a4"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.650307 21278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "51364830c0c4420eb3e075f77b619533"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "85b5064122aa649be9d7a702d4f78e54"
server_key_iv: "375eecd19844b08dd44332fd02b040a4"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.651496 21278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:03:35.652143 21293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.652298 21278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:35.652391 21278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "51364830c0c4420eb3e075f77b619533"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "85b5064122aa649be9d7a702d4f78e54"
server_key_iv: "375eecd19844b08dd44332fd02b040a4"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.652463 21278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:35.666597 21278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:35.668869 21278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:35.669024 21278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:35.669243 21278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:35.669564 21278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:35.669616 21278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.669648 21278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:35.669683 21278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.675136 21278 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:38565
I20260620 23:03:35.675220 21406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:38565 every 8 connection(s)
I20260620 23:03:35.675487 21278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:35.684248 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.676150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:41047 (local address 127.17.188.126:35223)
0620 23:03:35.676218 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:35.676220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:35.676349 (+ 129us) server_negotiation.cc:407] Connection header received
0620 23:03:35.676744 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:35.676744 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:35.676756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:35.676803 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:35.677652 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.678506 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.679413 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.679783 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.679842 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:35.681756 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:35.681757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:35.681758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:35.681766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:35.683038 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:35.683401 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:35.683402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:35.683403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:35.683467 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:35.683691 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:35.683692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:35.683692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:35.683791 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:35.683883 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:35.684103 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 23:03:35.684187 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:03:35.684345 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 21278
I20260620 23:03:35.684444 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:35.684446 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.676112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35223 (local address 127.17.188.65:41047)
0620 23:03:35.676243 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:35.676273 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:35.676672 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:35.676848 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:35.676851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:35.677378 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:35.677565 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:35.677569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.678586 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.678587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:35.679330 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:35.679332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.679852 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.679853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:35.679940 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:35.680288 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:35.680306 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:35.681655 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:35.683102 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:35.683107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:35.683109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:35.683312 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:35.683537 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:35.683538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:35.683538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:35.683635 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:35.683875 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:35.683879 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:35.683994 (+ 115us) client_negotiation.cc:769] Sending connection context
0620 23:03:35.684072 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 23:03:35.684207 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260620 23:03:35.684654 18161 external_mini_cluster.cc:1428] Setting key af9f2c6b08804eb1c3fd8d28fedda47e
I20260620 23:03:35.684855 21407 heartbeater.cc:347] Connected to a master server at 127.17.188.126:35223
I20260620 23:03:35.684952 21407 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:35.685096 21407 heartbeater.cc:511] Master 127.17.188.126:35223 requested a full tablet report, sending...
I20260620 23:03:35.685591 21216 ts_manager.cc:194] Registered new tserver with Master: 51364830c0c4420eb3e075f77b619533 (127.17.188.65:38565)
I20260620 23:03:35.686200 21216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:41047
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:03:35.706341 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35223
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:35.782426 21412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:35.782671 21412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:35.782711 21412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:35.783887 21412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:35.783955 21412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:35.783998 21412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:35.785466 21412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:35.785722 21412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:35.785944 21412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:35.790830 21417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:35.790867 21420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:35.790867 21418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:35.790972 21412 server_base.cc:1034] running on GCE node
I20260620 23:03:35.791231 21412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:35.791465 21412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:35.792630 21412 hybrid_clock.cc:648] HybridClock initialized: now 1781996615792610 us; error 31 us; skew 500 ppm
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:35.795015 21412 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:03:35.795437 21412 webserver.cc:460] Webserver started at http://127.17.188.66:39845/ using document root <none> and password file <none>
I20260620 23:03:35.795619 21412 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:35.795660 21412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:35.795743 21412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:35.796756 21412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "9e9bfcccfaa046e6a152b4f473e5eb63"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "122adb967c528a0100375132ac4f7272"
server_key_iv: "af9d1735cb21b9128b066369460a66b1"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.797057 21412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "9e9bfcccfaa046e6a152b4f473e5eb63"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "122adb967c528a0100375132ac4f7272"
server_key_iv: "af9d1735cb21b9128b066369460a66b1"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.798336 21412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:35.798964 21427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.799120 21412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:35.799201 21412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "9e9bfcccfaa046e6a152b4f473e5eb63"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "122adb967c528a0100375132ac4f7272"
server_key_iv: "af9d1735cb21b9128b066369460a66b1"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.799252 21412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:35.828451 21412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:35.830655 21412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:35.830756 21412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:35.830937 21412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:35.831164 21412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:35.831192 21412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.831213 21412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:35.831233 21412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.836582 21412 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:34459
I20260620 23:03:35.836658 21540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:34459 every 8 connection(s)
I20260620 23:03:35.836957 21412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:35.840516 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 21412
I20260620 23:03:35.840658 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:35.840844 18161 external_mini_cluster.cc:1428] Setting key 3800f1bc5678a02b2a1d7b1886655858
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:35.846513 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.837667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:35189 (local address 127.17.188.126:35223)
0620 23:03:35.837753 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:35.837755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:35.837941 (+ 186us) server_negotiation.cc:407] Connection header received
0620 23:03:35.838458 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:35.838459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:35.838470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:35.838513 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:35.839407 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.839932 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.841105 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.841389 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.841462 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:35.843629 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:35.843631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:35.843631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:35.843641 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:35.845088 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:35.845541 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:35.845543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:35.845543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:35.845604 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:35.845857 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:35.845857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:35.845858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:35.845984 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:35.846081 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:35.846340 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 23:03:35.846422 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:03:35.846751 21543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.837631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35223 (local address 127.17.188.66:35189)
0620 23:03:35.837823 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:35.837855 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:35.838342 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:35.838594 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:35.838597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:35.839120 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:35.839293 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:35.839298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.840042 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.840043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:35.841018 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:35.841020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.841470 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.841471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:35.841599 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:35.842064 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:35.842071 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:35.843512 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:35.845172 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:35.845179 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:35.845181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:35.845467 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:35.845672 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:35.845673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:35.845674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:35.845783 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:35.846076 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:35.846082 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:35.846215 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 23:03:35.846320 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 23:03:35.846469 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260620 23:03:35.847170 21541 heartbeater.cc:347] Connected to a master server at 127.17.188.126:35223
I20260620 23:03:35.847256 21541 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:35.847433 21541 heartbeater.cc:511] Master 127.17.188.126:35223 requested a full tablet report, sending...
I20260620 23:03:35.847775 21216 ts_manager.cc:194] Registered new tserver with Master: 9e9bfcccfaa046e6a152b4f473e5eb63 (127.17.188.66:34459)
I20260620 23:03:35.848162 21216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:35189
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:03:35.863096 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35223
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:35.938915 21546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:35.939095 21546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:35.939117 21546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:35.940217 21546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:35.940254 21546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:35.940306 21546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:35.941684 21546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:35.941884 21546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:35.942087 21546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:35.946564 21554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:35.946782 21551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:35.946770 21552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:35.946913 21546 server_base.cc:1034] running on GCE node
I20260620 23:03:35.947114 21546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:35.947333 21546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:35.948486 21546 hybrid_clock.cc:648] HybridClock initialized: now 1781996615948480 us; error 39 us; skew 500 ppm
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:35.950773 21546 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:03:35.951186 21546 webserver.cc:460] Webserver started at http://127.17.188.67:33463/ using document root <none> and password file <none>
I20260620 23:03:35.951387 21546 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:35.951448 21546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:35.951581 21546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:35.952489 21546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "099ca164d19d44cbac0df64ea4678123"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "ee464c5404713cf175defd0d3186b956"
server_key_iv: "34337a84b886bff0d9903ed8b618537f"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.952843 21546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "099ca164d19d44cbac0df64ea4678123"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "ee464c5404713cf175defd0d3186b956"
server_key_iv: "34337a84b886bff0d9903ed8b618537f"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.954299 21546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:35.955035 21561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.955204 21546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:35.955296 21546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "099ca164d19d44cbac0df64ea4678123"
format_stamp: "Formatted at 2026-06-20 23:03:35 on dist-test-slave-zpss"
server_key: "ee464c5404713cf175defd0d3186b956"
server_key_iv: "34337a84b886bff0d9903ed8b618537f"
server_key_version: "encryptionkey@0"
I20260620 23:03:35.955366 21546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:35.967861 21546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:35.969543 21546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:35.969681 21546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:35.969880 21546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:35.970136 21546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:35.970182 21546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.970213 21546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:35.970238 21546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:35.975704 21546 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:39875
I20260620 23:03:35.975750 21674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:39875 every 8 connection(s)
I20260620 23:03:35.976063 21546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:35.976991 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 21546
I20260620 23:03:35.977118 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:35.977291 18161 external_mini_cluster.cc:1428] Setting key c46c667e2e5b16db5ff4d7271bac937c
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:35.984445 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.976754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:56401 (local address 127.17.188.126:35223)
0620 23:03:35.976794 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:35.976797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:35.976957 (+ 160us) server_negotiation.cc:407] Connection header received
0620 23:03:35.977630 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:35.977632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:35.977643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:35.977702 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:35.978708 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.979182 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.980073 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.980240 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.980287 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:35.982176 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:35.982177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:35.982177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:35.982182 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:35.983305 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:35.983660 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:35.983660 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:35.983661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:35.983708 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:35.983910 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:35.983910 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:35.983910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:35.983999 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:35.984045 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:35.984335 (+ 290us) server_negotiation.cc:299] Negotiation successful
0620 23:03:35.984387 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:03:35.984663 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.976723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35223 (local address 127.17.188.67:56401)
0620 23:03:35.976841 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:35.976881 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:35.977523 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:35.977767 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:35.977771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:35.978455 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:35.978633 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:35.978638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.979255 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.979256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:35.980005 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:35.980006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:35.980304 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:35.980305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:35.980393 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:35.980861 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:35.980870 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:35.982105 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:35.983370 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:35.983375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:35.983377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:35.983598 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:35.983763 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:35.983763 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:35.983763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:35.983860 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:35.984067 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:35.984070 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:35.984201 (+ 131us) client_negotiation.cc:769] Sending connection context
0620 23:03:35.984311 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 23:03:35.984420 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":57,"threads_started":1}
I20260620 23:03:35.985110 21675 heartbeater.cc:347] Connected to a master server at 127.17.188.126:35223
I20260620 23:03:35.985213 21675 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:35.985417 21675 heartbeater.cc:511] Master 127.17.188.126:35223 requested a full tablet report, sending...
I20260620 23:03:35.985783 21216 ts_manager.cc:194] Registered new tserver with Master: 099ca164d19d44cbac0df64ea4678123 (127.17.188.67:39875)
I20260620 23:03:35.986097 21216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:56401
I20260620 23:03:35.988967 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 23:03:35 dist-test-slave-zpss krb5kdc[21171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 23:03:36 dist-test-slave-zpss krb5kdc[21171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996615, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:36.009227 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:36.003234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53116 (local address 127.17.188.126:35223)
0620 23:03:36.003283 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:36.003285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:36.003373 (+ 88us) server_negotiation.cc:407] Connection header received
0620 23:03:36.003406 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:36.003406 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:36.003413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:36.003457 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:36.003880 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:36.004308 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:36.004943 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:36.005097 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:36.005140 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:36.007047 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:36.007048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:36.007048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:36.007053 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:36.008395 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:36.008660 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:36.008660 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:36.008661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:36.008706 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:36.008841 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:36.008842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:36.008842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:36.008951 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:36.009018 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:36.009098 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:03:36.009148 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260620 23:03:36.009469 21216 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53116
I20260620 23:03:36.010005 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 21278
I20260620 23:03:36.014400 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 21412
I20260620 23:03:36.019200 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 21546
I20260620 23:03:36.023175 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 21184
2026-06-20T23:03:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1792 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:36 dist-test-slave-zpss krb5kdc[21687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:36 dist-test-slave-zpss krb5kdc[21687](info): set up 2 sockets
Jun 20 23:03:36 dist-test-slave-zpss krb5kdc[21687](info): commencing operation
krb5kdc: starting...
W20260620 23:03:37.049010 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:37.157850 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:41049
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:41049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:37.235255 21700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:37.235443 21700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:37.235464 21700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:37.236624 21700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:37.236663 21700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:37.236676 21700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:37.236730 21700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:37.236757 21700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:37.238185 21700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:41049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:41049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:37.238420 21700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:37.238646 21700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:03:37.243462 21700 server_base.cc:1034] running on GCE node
W20260620 23:03:37.243539 21708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:37.243546 21706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:37.243604 21705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:37.243907 21700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:37.244175 21700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:37.245357 21700 hybrid_clock.cc:648] HybridClock initialized: now 1781996617245333 us; error 33 us; skew 500 ppm
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:37.250865 21700 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:37.251343 21700 webserver.cc:460] Webserver started at http://127.17.188.126:37083/ using document root <none> and password file <none>
I20260620 23:03:37.251546 21700 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:37.251609 21700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:37.251754 21700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:37.252732 21700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "4682bad424fb41a8bfb93f92fd1f3a96"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "a15b11469d0d0a615f3598b5d74ceca8"
server_key_iv: "1d33b0994eddd2ba882558345af00811"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.253082 21700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "4682bad424fb41a8bfb93f92fd1f3a96"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "a15b11469d0d0a615f3598b5d74ceca8"
server_key_iv: "1d33b0994eddd2ba882558345af00811"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.254424 21700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:37.255091 21715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.255250 21700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:37.255349 21700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "4682bad424fb41a8bfb93f92fd1f3a96"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "a15b11469d0d0a615f3598b5d74ceca8"
server_key_iv: "1d33b0994eddd2ba882558345af00811"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.255431 21700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:37.272115 21700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:37.273823 21700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:37.273973 21700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:37.277498 21700 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:41049
I20260620 23:03:37.277542 21767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:41049 every 8 connection(s)
I20260620 23:03:37.278028 21700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:37.278777 21768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:37.280743 21768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96: Bootstrap starting.
I20260620 23:03:37.281239 21768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:37.281554 21768 log.cc:824] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:37.281621 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 21700
I20260620 23:03:37.281757 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:37.281924 18161 external_mini_cluster.cc:1428] Setting key 8b713b6cb727204b751fb29ffd66c682
I20260620 23:03:37.282258 21768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96: No bootstrap required, opened a new log
I20260620 23:03:37.284950 21768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4682bad424fb41a8bfb93f92fd1f3a96" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41049 } }
I20260620 23:03:37.285107 21768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:37.285153 21768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4682bad424fb41a8bfb93f92fd1f3a96, State: Initialized, Role: FOLLOWER
I20260620 23:03:37.285362 21768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4682bad424fb41a8bfb93f92fd1f3a96" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41049 } }
I20260620 23:03:37.285452 21768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:37.285495 21768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:37.285542 21768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:37.286027 21768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4682bad424fb41a8bfb93f92fd1f3a96" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41049 } }
I20260620 23:03:37.286139 21768 leader_election.cc:302] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4682bad424fb41a8bfb93f92fd1f3a96; no voters:
I20260620 23:03:37.286339 21768 leader_election.cc:288] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:37.286362 21773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:37.286460 21773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [term 1 LEADER]: Becoming Leader. State: Replica: 4682bad424fb41a8bfb93f92fd1f3a96, State: Running, Role: LEADER
I20260620 23:03:37.286600 21768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:37.286616 21773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4682bad424fb41a8bfb93f92fd1f3a96" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41049 } }
I20260620 23:03:37.287050 21774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4682bad424fb41a8bfb93f92fd1f3a96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4682bad424fb41a8bfb93f92fd1f3a96" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41049 } } }
I20260620 23:03:37.287071 21775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4682bad424fb41a8bfb93f92fd1f3a96. Latest consensus state: current_term: 1 leader_uuid: "4682bad424fb41a8bfb93f92fd1f3a96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4682bad424fb41a8bfb93f92fd1f3a96" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41049 } } }
I20260620 23:03:37.287168 21774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:37.287410 21775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:37.287577 21782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:37.288085 21782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:37.289765 21782 catalog_manager.cc:1347] Generated new cluster ID: 16c6a7a058df4a358c3e82d999441b29
I20260620 23:03:37.289829 21782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:37.292900 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.282403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.17.188.126:41049)
0620 23:03:37.282594 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:37.282603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:37.282639 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:03:37.283481 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:37.283487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:37.283529 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:37.283636 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:37.284428 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.285402 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.286290 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.286860 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.286963 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:37.289770 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:37.289773 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:37.289773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:37.289784 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:37.291436 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.291801 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.291804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.291804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.291911 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.292123 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.292124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.292124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.292347 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:37.292508 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:37.292608 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 23:03:37.292741 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
I20260620 23:03:37.300144 21782 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:37.300609 21782 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:37.304478 21782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4682bad424fb41a8bfb93f92fd1f3a96: Generated new TSK 0
I20260620 23:03:37.304637 21782 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:03:37.330277 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41049
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:37.424844 21794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:37.425038 21794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:37.425069 21794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:37.426221 21794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:37.426282 21794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:37.426324 21794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:37.427807 21794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:37.428045 21794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:37.428260 21794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:37.433919 21802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:37.433924 21800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:37.434057 21794 server_base.cc:1034] running on GCE node
W20260620 23:03:37.434100 21799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:37.434321 21794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:37.434577 21794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:37.435737 21794 hybrid_clock.cc:648] HybridClock initialized: now 1781996617435724 us; error 31 us; skew 500 ppm
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:37.438345 21794 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:03:37.438777 21794 webserver.cc:460] Webserver started at http://127.17.188.65:33073/ using document root <none> and password file <none>
I20260620 23:03:37.439010 21794 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:37.439077 21794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:37.439236 21794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:37.440287 21794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "477a69e9de014b55aae3640ff5bead04"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "ec5188ec886eb747b2ec7b3003e9ff0d"
server_key_iv: "a5c92c5b1f3157be4d34ece333f90fdf"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.440614 21794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "477a69e9de014b55aae3640ff5bead04"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "ec5188ec886eb747b2ec7b3003e9ff0d"
server_key_iv: "a5c92c5b1f3157be4d34ece333f90fdf"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.441939 21794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260620 23:03:37.442703 21809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.442935 21794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:37.443045 21794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "477a69e9de014b55aae3640ff5bead04"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "ec5188ec886eb747b2ec7b3003e9ff0d"
server_key_iv: "a5c92c5b1f3157be4d34ece333f90fdf"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.443122 21794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:37.457587 21794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:37.460062 21794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:37.460214 21794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:37.460424 21794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:37.460759 21794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:37.460801 21794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.460842 21794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:37.460868 21794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.468077 21794 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:33119
I20260620 23:03:37.468117 21922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:33119 every 8 connection(s)
I20260620 23:03:37.468415 21794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:37.475076 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 21794
I20260620 23:03:37.475195 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:37.475365 18161 external_mini_cluster.cc:1428] Setting key c67ba2c6a2449d6d98c6511a29c3d527
I20260620 23:03:37.477950 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.469146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:50267 (local address 127.17.188.126:41049)
0620 23:03:37.469206 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:37.469208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:37.469450 (+ 242us) server_negotiation.cc:407] Connection header received
0620 23:03:37.470348 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:37.470349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:37.470362 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:37.470411 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:37.470965 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.471490 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.472445 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.472662 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.472752 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:37.474764 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:37.474765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:37.474765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:37.474772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:37.476560 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.476957 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.476959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.476959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.477022 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.477251 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.477252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.477252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.477412 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:37.477487 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:37.477829 (+ 342us) server_negotiation.cc:299] Negotiation successful
0620 23:03:37.477891 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:03:37.478225 21925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.469134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:41049 (local address 127.17.188.65:50267)
0620 23:03:37.469291 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:37.469363 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:37.470236 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:37.470472 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:37.470476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:37.470708 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:37.470871 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:37.470875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.471587 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.471588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:37.472355 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:37.472357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.472761 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.472762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:37.472881 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:37.473365 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:37.473380 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:37.474660 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:37.476647 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:37.476653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:37.476655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:37.476880 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:37.477089 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:37.477089 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:37.477090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:37.477186 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:37.477490 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:37.477497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:37.477678 (+ 181us) client_negotiation.cc:769] Sending connection context
0620 23:03:37.477800 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 23:03:37.477989 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260620 23:03:37.478807 21923 heartbeater.cc:347] Connected to a master server at 127.17.188.126:41049
I20260620 23:03:37.478914 21923 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:37.479084 21923 heartbeater.cc:511] Master 127.17.188.126:41049 requested a full tablet report, sending...
I20260620 23:03:37.479645 21732 ts_manager.cc:194] Registered new tserver with Master: 477a69e9de014b55aae3640ff5bead04 (127.17.188.65:33119)
I20260620 23:03:37.480602 21732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:50267
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:03:37.498109 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41049
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:37.579654 21928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:37.579859 21928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:37.579918 21928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:37.580957 21928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:37.581004 21928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:37.581030 21928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:37.582544 21928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:37.582728 21928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:37.582887 21928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:37.587615 21936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:37.587661 21934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:37.587633 21933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:37.588023 21928 server_base.cc:1034] running on GCE node
I20260620 23:03:37.588217 21928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:37.588384 21928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:37.589512 21928 hybrid_clock.cc:648] HybridClock initialized: now 1781996617589503 us; error 29 us; skew 500 ppm
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:37.591734 21928 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:03:37.592087 21928 webserver.cc:460] Webserver started at http://127.17.188.66:33149/ using document root <none> and password file <none>
I20260620 23:03:37.592353 21928 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:37.592396 21928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:37.592550 21928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:37.593580 21928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "03606b8c83604228b91f4b5e7608f186"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "bfe81263fd6e3036c139fa7683d17f48"
server_key_iv: "746ce0a99140df0a8b6d8cd4edb1b2a6"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.593916 21928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "03606b8c83604228b91f4b5e7608f186"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "bfe81263fd6e3036c139fa7683d17f48"
server_key_iv: "746ce0a99140df0a8b6d8cd4edb1b2a6"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.595124 21928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:03:37.595760 21943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.595928 21928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:37.596004 21928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "03606b8c83604228b91f4b5e7608f186"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "bfe81263fd6e3036c139fa7683d17f48"
server_key_iv: "746ce0a99140df0a8b6d8cd4edb1b2a6"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.596052 21928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:37.609252 21928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:37.610949 21928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:37.611049 21928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:37.611202 21928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:37.611472 21928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:37.611503 21928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.611524 21928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:37.611570 21928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.616923 21928 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:38189
I20260620 23:03:37.617003 22056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:38189 every 8 connection(s)
I20260620 23:03:37.617274 21928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:37.622558 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 21928
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:37.622641 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:37.622835 18161 external_mini_cluster.cc:1428] Setting key 95c23849d7441a1ceb13d05ca9fb5562
I20260620 23:03:37.626060 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.617934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:53455 (local address 127.17.188.126:41049)
0620 23:03:37.617998 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:37.617999 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:37.618160 (+ 161us) server_negotiation.cc:407] Connection header received
0620 23:03:37.619050 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:37.619051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:37.619060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:37.619108 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:37.619648 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.620128 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.621032 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.621207 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.621254 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:37.623157 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:37.623158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:37.623158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:37.623165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:37.624721 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.625079 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.625081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.625081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.625143 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.625467 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.625468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.625468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.625576 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:37.625660 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:37.625900 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 23:03:37.625975 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:03:37.626237 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.617894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:41049 (local address 127.17.188.66:53455)
0620 23:03:37.618055 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:37.618091 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:37.618960 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:37.619166 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:37.619169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:37.619397 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:37.619567 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:37.619571 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.620218 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.620218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:37.620958 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:37.620960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.621272 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.621273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:37.621382 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:37.621774 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:37.621781 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:37.623077 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:37.624793 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:37.624798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:37.624800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:37.624985 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:37.625227 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:37.625228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:37.625229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:37.625393 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:37.625643 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:37.625648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:37.625778 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 23:03:37.625877 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 23:03:37.626016 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260620 23:03:37.626626 22057 heartbeater.cc:347] Connected to a master server at 127.17.188.126:41049
I20260620 23:03:37.626736 22057 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:37.626897 22057 heartbeater.cc:511] Master 127.17.188.126:41049 requested a full tablet report, sending...
I20260620 23:03:37.627300 21732 ts_manager.cc:194] Registered new tserver with Master: 03606b8c83604228b91f4b5e7608f186 (127.17.188.66:38189)
I20260620 23:03:37.627827 21732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:53455
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:03:37.646039 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41049
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:37.722822 22062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:37.723021 22062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:37.723042 22062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:37.724123 22062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:37.724160 22062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:37.724220 22062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:37.725627 22062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:37.725878 22062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:37.726063 22062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:37.730654 22067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:37.730604 22068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:37.730639 22070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:37.730834 22062 server_base.cc:1034] running on GCE node
I20260620 23:03:37.731109 22062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:37.731328 22062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:37.732458 22062 hybrid_clock.cc:648] HybridClock initialized: now 1781996617732449 us; error 25 us; skew 500 ppm
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:37.735109 22062 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:03:37.735507 22062 webserver.cc:460] Webserver started at http://127.17.188.67:38079/ using document root <none> and password file <none>
I20260620 23:03:37.735723 22062 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:37.735764 22062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:37.735891 22062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:37.736819 22062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "0ccc810206e249c094e5795876434788"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "adf2c7c1324b01514b083423e7e9dd4b"
server_key_iv: "5b277b038d78a3ab1840b77e1c4babf4"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.737155 22062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "0ccc810206e249c094e5795876434788"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "adf2c7c1324b01514b083423e7e9dd4b"
server_key_iv: "5b277b038d78a3ab1840b77e1c4babf4"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.738368 22062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:37.739044 22077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.739234 22062 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:37.739315 22062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "0ccc810206e249c094e5795876434788"
format_stamp: "Formatted at 2026-06-20 23:03:37 on dist-test-slave-zpss"
server_key: "adf2c7c1324b01514b083423e7e9dd4b"
server_key_iv: "5b277b038d78a3ab1840b77e1c4babf4"
server_key_version: "encryptionkey@0"
I20260620 23:03:37.739377 22062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:37.757069 22062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:37.758811 22062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:37.758951 22062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:37.759140 22062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:37.759407 22062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:37.759438 22062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.759474 22062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:37.759500 22062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:37.764612 22062 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:41995
I20260620 23:03:37.764675 22190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:41995 every 8 connection(s)
I20260620 23:03:37.764937 22062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:37.770376 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 22062
I20260620 23:03:37.770484 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:37.770680 18161 external_mini_cluster.cc:1428] Setting key 87d8edeb18612b7b61221e09cdc3f761
I20260620 23:03:37.773234 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.765558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:56037 (local address 127.17.188.126:41049)
0620 23:03:37.765642 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:37.765645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:37.765790 (+ 145us) server_negotiation.cc:407] Connection header received
0620 23:03:37.766628 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:37.766629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:37.766639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:37.766695 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:37.767197 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.767690 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.768539 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.768734 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.768781 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:37.770709 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:37.770710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:37.770710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:37.770718 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:37.772222 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.772565 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.772565 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.772565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.772611 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.772798 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.772798 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.772798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.772889 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:37.772936 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:37.773117 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 23:03:37.773167 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 23:03:37.773442 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.765515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:41049 (local address 127.17.188.67:56037)
0620 23:03:37.765693 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:37.765732 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:37.766544 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:37.766747 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:37.766750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:37.766970 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:37.767134 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:37.767137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.767757 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.767757 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:37.768475 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:37.768477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.768797 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.768798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:37.768885 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:37.769264 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:37.769271 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:37.770629 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:37.772288 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:37.772293 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:37.772295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:37.772504 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:37.772664 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:37.772665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:37.772665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:37.772748 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:37.772939 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:37.772943 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:37.773035 (+ 92us) client_negotiation.cc:769] Sending connection context
0620 23:03:37.773108 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:03:37.773217 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260620 23:03:37.773871 22191 heartbeater.cc:347] Connected to a master server at 127.17.188.126:41049
I20260620 23:03:37.773973 22191 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:37.774132 22191 heartbeater.cc:511] Master 127.17.188.126:41049 requested a full tablet report, sending...
I20260620 23:03:37.774482 21732 ts_manager.cc:194] Registered new tserver with Master: 0ccc810206e249c094e5795876434788 (127.17.188.67:41995)
I20260620 23:03:37.774812 21732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:56037
I20260620 23:03:37.782436 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:37.803932 22198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.796649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37240 (local address 127.17.188.65:33119)
0620 23:03:37.796798 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:37.796803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:37.796814 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:03:37.796846 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:37.796848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:37.796860 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:37.796922 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:37.797442 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.798116 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.798888 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.799192 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.799246 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:37.801427 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:37.801433 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:37.801434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:37.801440 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:37.802963 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.803260 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.803262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.803262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.803315 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.803466 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.803466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.803466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.803582 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:37.803685 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:37.803736 (+ 51us) server_negotiation.cc:299] Negotiation successful
0620 23:03:37.803794 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:37.810660 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.805067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.17.188.126:41049)
0620 23:03:37.805106 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:37.805108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:37.805146 (+ 38us) server_negotiation.cc:407] Connection header received
0620 23:03:37.805209 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:37.805210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:37.805217 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:37.805260 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:37.805713 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.806120 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.806664 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:37.806818 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:37.806859 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:37.808639 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:37.808658 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:37.808658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:37.808664 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:37.809843 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.810113 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.810113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.810114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.810157 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:37.810292 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:37.810292 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:37.810292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:37.810417 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:37.810490 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:37.810559 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:03:37.810609 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 23:03:37.810889 21732 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33852
I20260620 23:03:37.811213 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 21794
I20260620 23:03:37.816144 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 21928
I20260620 23:03:37.820011 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 22062
I20260620 23:03:37.823714 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 21700
2026-06-20T23:03:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1800 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[22203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[22203](info): set up 2 sockets
Jun 20 23:03:37 dist-test-slave-zpss krb5kdc[22203](info): commencing operation
krb5kdc: starting...
W20260620 23:03:38.849234 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:38 dist-test-slave-zpss krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:38.957971 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:41141
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:41141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:39.033044 22216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:39.033243 22216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:39.033265 22216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:39.034399 22216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:39.034438 22216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:39.034452 22216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:39.034466 22216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:39.034477 22216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:39.036077 22216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:41141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:41141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:39.036252 22216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:39.036405 22216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:39.040900 22222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:39.040925 22221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.040952 22216 server_base.cc:1034] running on GCE node
W20260620 23:03:39.040925 22224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.041395 22216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:39.041639 22216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:39.042781 22216 hybrid_clock.cc:648] HybridClock initialized: now 1781996619042769 us; error 27 us; skew 500 ppm
Jun 20 23:03:39 dist-test-slave-zpss krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:39.045029 22216 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:39.045428 22216 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:39.045493 22216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:39.045621 22216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:39.046586 22216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "7028b8354b504a14a4123796fcf5e11d"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "98e1c45b3a3485bca976547123352587"
server_key_iv: "f1e37023f427d1594ee4286e1e64b892"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.046957 22216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "7028b8354b504a14a4123796fcf5e11d"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "98e1c45b3a3485bca976547123352587"
server_key_iv: "f1e37023f427d1594ee4286e1e64b892"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.048187 22216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:39.048877 22230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.049046 22216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:39.049139 22216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "7028b8354b504a14a4123796fcf5e11d"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "98e1c45b3a3485bca976547123352587"
server_key_iv: "f1e37023f427d1594ee4286e1e64b892"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.049220 22216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:39.057009 22216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:39.058728 22216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:39.058895 22216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:39.062804 22216 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:41141
I20260620 23:03:39.062850 22282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:41141 every 8 connection(s)
I20260620 23:03:39.063113 22216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:39.063983 22283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:39.066227 22283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d: Bootstrap starting.
I20260620 23:03:39.066815 22283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:39.067111 22283 log.cc:824] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:39.067723 22283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d: No bootstrap required, opened a new log
I20260620 23:03:39.069236 22283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7028b8354b504a14a4123796fcf5e11d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41141 } }
I20260620 23:03:39.069393 22283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:39.069416 22283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7028b8354b504a14a4123796fcf5e11d, State: Initialized, Role: FOLLOWER
I20260620 23:03:39.069577 22283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7028b8354b504a14a4123796fcf5e11d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41141 } }
I20260620 23:03:39.069657 22283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:39.069702 22283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:39.069751 22283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:39.070567 22283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7028b8354b504a14a4123796fcf5e11d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41141 } }
I20260620 23:03:39.070675 22283 leader_election.cc:302] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7028b8354b504a14a4123796fcf5e11d; no voters:
I20260620 23:03:39.070853 22283 leader_election.cc:288] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:39.070914 22286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:39.071079 22286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [term 1 LEADER]: Becoming Leader. State: Replica: 7028b8354b504a14a4123796fcf5e11d, State: Running, Role: LEADER
I20260620 23:03:39.071099 22283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:39.071202 22286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7028b8354b504a14a4123796fcf5e11d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41141 } }
I20260620 23:03:39.071578 22287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7028b8354b504a14a4123796fcf5e11d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7028b8354b504a14a4123796fcf5e11d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41141 } } }
I20260620 23:03:39.071591 22288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7028b8354b504a14a4123796fcf5e11d. Latest consensus state: current_term: 1 leader_uuid: "7028b8354b504a14a4123796fcf5e11d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7028b8354b504a14a4123796fcf5e11d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41141 } } }
I20260620 23:03:39.071774 22288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:39.071943 22287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:39.072091 22295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:39.072244 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 22216
I20260620 23:03:39.072364 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:39.072554 18161 external_mini_cluster.cc:1428] Setting key b2cbee71101eaf96835c7e5b091f0fad
I20260620 23:03:39.072567 22295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:39.074222 22295 catalog_manager.cc:1347] Generated new cluster ID: ad1fd6395a6546ca8db7d45a8f63f897
I20260620 23:03:39.074291 22295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:39.084547 22295 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:39.085219 22295 catalog_manager.cc:1504] Loading token signing keys...
Jun 20 23:03:39 dist-test-slave-zpss krb5kdc[22203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:39.093186 22295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d: Generated new TSK 0
I20260620 23:03:39.093405 22295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:03:39.093602 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.073001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.17.188.126:41141)
0620 23:03:39.073161 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.073170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.084415 (+ 11245us) server_negotiation.cc:407] Connection header received
0620 23:03:39.084576 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.084583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.084642 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.084799 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:39.085425 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.086272 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.086990 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.087394 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.087508 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.089894 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:39.089898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:39.089898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:39.089908 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:39.091564 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.091940 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.091945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.091949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.092027 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.092239 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.092240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.092240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.092454 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:39.092607 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.092712 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.093038 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:03:39.122246 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41141
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:39.235430 22309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:39.235615 22309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:39.235636 22309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:39.236791 22309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:39.236826 22309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:39.236851 22309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:39.238359 22309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:39.238597 22309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:39.238837 22309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:39.244506 22315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:39.244541 22314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:39.244506 22317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.244666 22309 server_base.cc:1034] running on GCE node
I20260620 23:03:39.244947 22309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:39.245174 22309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:39.246321 22309 hybrid_clock.cc:648] HybridClock initialized: now 1781996619246324 us; error 39 us; skew 500 ppm
Jun 20 23:03:39 dist-test-slave-zpss krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:39.248700 22309 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:03:39.249073 22309 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:39.249140 22309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:39.249276 22309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:39.250311 22309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "7369052f7cd94740a3db31cccdbffccd"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "a8ff0356d43dd833291df82ec3aecae4"
server_key_iv: "ac1606e54bf74e193d3276f50f626e04"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.250666 22309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "7369052f7cd94740a3db31cccdbffccd"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "a8ff0356d43dd833291df82ec3aecae4"
server_key_iv: "ac1606e54bf74e193d3276f50f626e04"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.252033 22309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:03:39.252718 22323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.252892 22309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:39.252988 22309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "7369052f7cd94740a3db31cccdbffccd"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "a8ff0356d43dd833291df82ec3aecae4"
server_key_iv: "ac1606e54bf74e193d3276f50f626e04"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.253072 22309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:39.272217 22309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:39.274605 22309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:39.274760 22309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:39.274981 22309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:39.275293 22309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:39.275346 22309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.275385 22309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:39.275413 22309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.280979 22309 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:46345
I20260620 23:03:39.281028 22436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:46345 every 8 connection(s)
I20260620 23:03:39.281301 22309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:03:39 dist-test-slave-zpss krb5kdc[22203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:39.287204 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 22309
I20260620 23:03:39.287303 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:39.287489 18161 external_mini_cluster.cc:1428] Setting key 82d5297cfe17f2190337d204e984e0ce
I20260620 23:03:39.290104 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.281991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:39833 (local address 127.17.188.126:41141)
0620 23:03:39.282063 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.282065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.282258 (+ 193us) server_negotiation.cc:407] Connection header received
0620 23:03:39.282688 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.282689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.282697 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.282753 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:39.283601 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.284083 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.284972 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.285149 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.285196 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.287219 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:39.287221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:39.287221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:39.287227 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:39.288700 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.289126 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.289127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.289127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.289182 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.289479 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.289480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.289481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.289596 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:39.289670 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.289924 (+ 254us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.290006 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 23:03:39.290285 22439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.281986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:41141 (local address 127.17.188.65:39833)
0620 23:03:39.282134 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.282172 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:39.282605 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:39.282803 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:39.282806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:39.283043 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:39.283508 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.283512 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.284167 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.284167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.284900 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.284901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.285211 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.285211 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.285365 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.285863 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:39.285871 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:39.287124 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:39.288823 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:39.288829 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:39.288831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:39.289049 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:39.289253 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:39.289253 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:39.289254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:39.289410 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:39.289661 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:39.289667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:39.289797 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 23:03:39.289903 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 23:03:39.290041 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260620 23:03:39.290688 22437 heartbeater.cc:347] Connected to a master server at 127.17.188.126:41141
I20260620 23:03:39.290787 22437 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:39.290944 22437 heartbeater.cc:511] Master 127.17.188.126:41141 requested a full tablet report, sending...
I20260620 23:03:39.291473 22247 ts_manager.cc:194] Registered new tserver with Master: 7369052f7cd94740a3db31cccdbffccd (127.17.188.65:46345)
I20260620 23:03:39.292425 22247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:39833
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:03:39.309440 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41141
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:39.389482 22442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:39.389757 22442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:39.389801 22442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:39.391002 22442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:39.391080 22442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:39.391126 22442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:39.392660 22442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:39.392946 22442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:39.393198 22442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:39.398211 22450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:39.398178 22447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.398291 22442 server_base.cc:1034] running on GCE node
W20260620 23:03:39.398197 22448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.398579 22442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:39.398806 22442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:39.399952 22442 hybrid_clock.cc:648] HybridClock initialized: now 1781996619399944 us; error 28 us; skew 500 ppm
Jun 20 23:03:39 dist-test-slave-zpss krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:39.402355 22442 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:03:39.402721 22442 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:39.402789 22442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:39.402921 22442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:39.403934 22442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "bc5d3c4d61d6497ea743f791d9642b8b"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "3054169595c57c7763849232e3730d4c"
server_key_iv: "86835ab79fea7610cf340e051e74a880"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.404291 22442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "bc5d3c4d61d6497ea743f791d9642b8b"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "3054169595c57c7763849232e3730d4c"
server_key_iv: "86835ab79fea7610cf340e051e74a880"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.405735 22442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:39.406411 22456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.406572 22442 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.406665 22442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "bc5d3c4d61d6497ea743f791d9642b8b"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "3054169595c57c7763849232e3730d4c"
server_key_iv: "86835ab79fea7610cf340e051e74a880"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.406744 22442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:39.428965 22442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:39.431289 22442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:39.431455 22442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:39.431671 22442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:39.431960 22442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:39.432020 22442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.432053 22442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:39.432080 22442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.437598 22442 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:45233
I20260620 23:03:39.437636 22569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:45233 every 8 connection(s)
I20260620 23:03:39.437877 22442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:03:39 dist-test-slave-zpss krb5kdc[22203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:39.443641 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 22442
I20260620 23:03:39.443776 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:39.443977 18161 external_mini_cluster.cc:1428] Setting key 1a7e3cbfbfef565d49aeb818c9592766
I20260620 23:03:39.446853 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.438505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:38873 (local address 127.17.188.126:41141)
0620 23:03:39.438581 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.438583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.438773 (+ 190us) server_negotiation.cc:407] Connection header received
0620 23:03:39.439212 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.439213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.439222 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.439274 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:39.440145 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.440653 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.441648 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.441828 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.441875 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.443849 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:39.443850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:39.443850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:39.443858 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:39.445544 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.445916 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.445917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.445917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.445978 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.446290 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.446291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.446292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.446395 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:39.446447 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.446739 (+ 292us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.446795 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:03:39.447124 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.438484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:41141 (local address 127.17.188.66:38873)
0620 23:03:39.438659 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.438692 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:39.439135 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:39.439331 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:39.439334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:39.439561 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:39.440049 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.440067 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.440721 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.440722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.441581 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.441583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.441888 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.441889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.441973 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.442435 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:39.442442 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:39.443736 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:39.445640 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:39.445646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:39.445647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:39.445847 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:39.446063 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:39.446064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:39.446064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:39.446213 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:39.446457 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:39.446462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:39.446611 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 23:03:39.446722 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 23:03:39.446879 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260620 23:03:39.447546 22570 heartbeater.cc:347] Connected to a master server at 127.17.188.126:41141
I20260620 23:03:39.447628 22570 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:39.447784 22570 heartbeater.cc:511] Master 127.17.188.126:41141 requested a full tablet report, sending...
I20260620 23:03:39.448137 22247 ts_manager.cc:194] Registered new tserver with Master: bc5d3c4d61d6497ea743f791d9642b8b (127.17.188.66:45233)
I20260620 23:03:39.448643 22247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:38873
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:03:39.467553 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41141
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:39.545468 22575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:39.545660 22575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:39.545696 22575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:39.546842 22575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:39.546908 22575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:39.546952 22575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:39.548455 22575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:41141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:39.548707 22575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:39.548940 22575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:39.553718 22580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:39.553774 22583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.553928 22575 server_base.cc:1034] running on GCE node
W20260620 23:03:39.553975 22581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.554189 22575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:39.554426 22575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:39.555601 22575 hybrid_clock.cc:648] HybridClock initialized: now 1781996619555587 us; error 29 us; skew 500 ppm
Jun 20 23:03:39 dist-test-slave-zpss krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:39.557979 22575 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:03:39.558298 22575 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:39.558369 22575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:39.558518 22575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:39.559509 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "472a40375d944986b5ec78b19b99b19a"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "4dcb0b5becfff4a2af4c4fd5b53d543c"
server_key_iv: "f137b82b82015f270498929859606158"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.559870 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "472a40375d944986b5ec78b19b99b19a"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "4dcb0b5becfff4a2af4c4fd5b53d543c"
server_key_iv: "f137b82b82015f270498929859606158"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.561088 22575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:03:39.561873 22589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.562033 22575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.562129 22575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "472a40375d944986b5ec78b19b99b19a"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "4dcb0b5becfff4a2af4c4fd5b53d543c"
server_key_iv: "f137b82b82015f270498929859606158"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.562206 22575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:39.572737 22575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:39.574581 22575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:39.574694 22575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:39.574836 22575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:39.575070 22575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:39.575099 22575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.575119 22575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:39.575130 22575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.580667 22575 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:42205
I20260620 23:03:39.580732 22702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:42205 every 8 connection(s)
I20260620 23:03:39.580999 22575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:39.581804 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 22575
I20260620 23:03:39.581930 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:39.582114 18161 external_mini_cluster.cc:1428] Setting key 67e12171c6d5de88856665ff9f177e16
Jun 20 23:03:39 dist-test-slave-zpss krb5kdc[22203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:39.589743 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.581723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:35557 (local address 127.17.188.126:41141)
0620 23:03:39.581771 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.581774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.581930 (+ 156us) server_negotiation.cc:407] Connection header received
0620 23:03:39.582531 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.582532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.582544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.582606 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:39.583617 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.584104 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.585124 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.585313 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.585460 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.587303 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:39.587305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:39.587305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:39.587311 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:39.588650 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.588967 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.588967 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.588968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.589014 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.589204 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.589204 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.589205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.589299 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:39.589411 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.589626 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.589684 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:03:39.589910 22705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.581657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:41141 (local address 127.17.188.67:35557)
0620 23:03:39.581801 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.581839 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:39.582426 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:39.582661 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:39.582665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:39.583014 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:39.583522 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.583527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.584178 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.584179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.584950 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.584952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.585481 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.585482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.585573 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.585947 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:39.585954 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:39.587229 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:39.588714 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:39.588718 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:39.588720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:39.588907 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:39.589069 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:39.589069 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:39.589069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:39.589152 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:39.589415 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:39.589418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:39.589518 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 23:03:39.589597 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:03:39.589702 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260620 23:03:39.590440 22703 heartbeater.cc:347] Connected to a master server at 127.17.188.126:41141
I20260620 23:03:39.590543 22703 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:39.590677 22703 heartbeater.cc:511] Master 127.17.188.126:41141 requested a full tablet report, sending...
I20260620 23:03:39.591022 22247 ts_manager.cc:194] Registered new tserver with Master: 472a40375d944986b5ec78b19b99b19a (127.17.188.67:42205)
I20260620 23:03:39.591373 22247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:35557
I20260620 23:03:39.593701 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:39.600415 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.594949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37806 (local address 127.17.188.126:41141)
0620 23:03:39.595009 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.595011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.595100 (+ 89us) server_negotiation.cc:407] Connection header received
0620 23:03:39.595184 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.595185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.595196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.595237 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:39.595748 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.596213 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.596916 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.597106 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.597186 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.598225 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:39.598226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:39.598227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:39.598233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:39.599263 (+ 1030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.599610 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.599611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.599611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.599666 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:39.599862 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:39.599863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:39.599863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:39.600001 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:39.600103 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.600249 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.600319 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:03:39.601824 22247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:39.603237 22247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:39.609277 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.605431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:42205 (local address 127.0.0.1:41722)
0620 23:03:39.605908 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.605918 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:39.605976 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:39.606139 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:39.606140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:39.606141 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:39.606182 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.606183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.607321 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.607322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.608274 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.608276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.609081 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.609082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.609136 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.609145 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 23:03:39.609195 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 23:03:39.609229 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":35,"thread_start_us":86,"threads_started":1}
I20260620 23:03:39.609390 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.605732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.17.188.67:42205)
0620 23:03:39.605919 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.605922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.605984 (+ 62us) server_negotiation.cc:407] Connection header received
0620 23:03:39.606042 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.606044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.606058 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.606135 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:39.606259 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.607236 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.608357 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.609003 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.609075 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.609125 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.609220 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.609257 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
I20260620 23:03:39.609894 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.605428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:46345 (local address 127.0.0.1:48078)
0620 23:03:39.605538 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.605575 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:39.605627 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:39.605901 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:39.605903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:39.605905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:39.605967 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.605968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.607252 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.607254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.608498 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.608500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.609657 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.609658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.609729 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.609733 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:39.609797 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:03:39.609833 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":31,"thread_start_us":71,"threads_started":1}
I20260620 23:03:39.609942 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.605522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48078 (local address 127.17.188.65:46345)
0620 23:03:39.605652 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.605656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.605669 (+ 13us) server_negotiation.cc:407] Connection header received
0620 23:03:39.605707 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.605709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.605724 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.605804 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:39.606047 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.607154 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.608576 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.609261 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.609402 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.609577 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.609830 (+ 253us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.609866 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260620 23:03:39.610073 22637 tablet_service.cc:1467] Processing CreateTablet for tablet c315b5e980b949e48eb2b06eaa7174f8 (DEFAULT_TABLE table=test-table [id=fccb725481964c20a01afe5ded860fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:39.610401 22637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c315b5e980b949e48eb2b06eaa7174f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:39.610611 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.605638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:45233 (local address 127.0.0.1:52586)
0620 23:03:39.605965 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.606216 (+ 251us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:39.606255 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:39.606505 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:39.606505 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:39.606508 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:39.606546 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.606548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.607370 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.607370 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.608498 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.608500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.610450 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.610451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.610508 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.610513 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:03:39.610542 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 23:03:39.610569 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":67,"thread_start_us":195,"threads_started":1}
I20260620 23:03:39.610771 22371 tablet_service.cc:1467] Processing CreateTablet for tablet c315b5e980b949e48eb2b06eaa7174f8 (DEFAULT_TABLE table=test-table [id=fccb725481964c20a01afe5ded860fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:39.610893 22719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.605779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.17.188.66:45233)
0620 23:03:39.606363 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.606367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.606375 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:39.606402 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.606404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.606415 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.606494 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:39.606598 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.607265 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.609611 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.610356 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.610720 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.610780 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.610804 (+ 24us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.610838 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":71,"threads_started":1}
I20260620 23:03:39.611047 22371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c315b5e980b949e48eb2b06eaa7174f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:39.611388 22504 tablet_service.cc:1467] Processing CreateTablet for tablet c315b5e980b949e48eb2b06eaa7174f8 (DEFAULT_TABLE table=test-table [id=fccb725481964c20a01afe5ded860fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:39.611593 22504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c315b5e980b949e48eb2b06eaa7174f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:39.613684 22720 tablet_bootstrap.cc:492] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: Bootstrap starting.
I20260620 23:03:39.614187 22721 tablet_bootstrap.cc:492] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: Bootstrap starting.
I20260620 23:03:39.614334 22720 tablet_bootstrap.cc:654] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:39.614727 22722 tablet_bootstrap.cc:492] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd: Bootstrap starting.
I20260620 23:03:39.614823 22721 tablet_bootstrap.cc:654] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:39.614900 22720 log.cc:824] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:39.615417 22721 log.cc:824] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:39.615459 22722 tablet_bootstrap.cc:654] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:39.615711 22720 tablet_bootstrap.cc:492] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: No bootstrap required, opened a new log
I20260620 23:03:39.615788 22720 ts_tablet_manager.cc:1397] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:03:39.616019 22722 log.cc:824] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:39.616137 22721 tablet_bootstrap.cc:492] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: No bootstrap required, opened a new log
I20260620 23:03:39.616204 22721 ts_tablet_manager.cc:1397] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:03:39.616736 22722 tablet_bootstrap.cc:492] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd: No bootstrap required, opened a new log
I20260620 23:03:39.616813 22722 ts_tablet_manager.cc:1397] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:03:39.617717 22720 raft_consensus.cc:348] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } }
I20260620 23:03:39.617889 22720 raft_consensus.cc:374] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:39.617921 22720 raft_consensus.cc:729] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 472a40375d944986b5ec78b19b99b19a, State: Initialized, Role: FOLLOWER
I20260620 23:03:39.618057 22720 consensus_queue.cc:260] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } }
I20260620 23:03:39.618090 22721 raft_consensus.cc:348] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } }
I20260620 23:03:39.618249 22721 raft_consensus.cc:374] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:39.618300 22721 raft_consensus.cc:729] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5d3c4d61d6497ea743f791d9642b8b, State: Initialized, Role: FOLLOWER
I20260620 23:03:39.618338 22720 ts_tablet_manager.cc:1428] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:03:39.618460 22703 heartbeater.cc:503] Master 127.17.188.126:41141 was elected leader, sending a full tablet report...
I20260620 23:03:39.618439 22721 consensus_queue.cc:260] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } }
I20260620 23:03:39.618382 22722 raft_consensus.cc:348] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } }
I20260620 23:03:39.618530 22722 raft_consensus.cc:374] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:39.618559 22722 raft_consensus.cc:729] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7369052f7cd94740a3db31cccdbffccd, State: Initialized, Role: FOLLOWER
I20260620 23:03:39.618650 22722 consensus_queue.cc:260] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } }
I20260620 23:03:39.618697 22721 ts_tablet_manager.cc:1428] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 23:03:39.618795 22570 heartbeater.cc:503] Master 127.17.188.126:41141 was elected leader, sending a full tablet report...
I20260620 23:03:39.618887 22722 ts_tablet_manager.cc:1428] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:03:39.619017 22437 heartbeater.cc:503] Master 127.17.188.126:41141 was elected leader, sending a full tablet report...
I20260620 23:03:39.646045 22728 raft_consensus.cc:482] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:39.646133 22728 raft_consensus.cc:504] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } }
I20260620 23:03:39.646440 22728 leader_election.cc:288] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 472a40375d944986b5ec78b19b99b19a (127.17.188.67:42205), bc5d3c4d61d6497ea743f791d9642b8b (127.17.188.66:45233)
I20260620 23:03:39.650141 22719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.646694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:51627 (local address 127.17.188.66:45233)
0620 23:03:39.646737 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.646738 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.646857 (+ 119us) server_negotiation.cc:407] Connection header received
0620 23:03:39.646885 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.646886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.646894 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.646942 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:39.647099 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.647759 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.649233 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.649806 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.649884 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.649893 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.650023 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.650066 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:03:39.650154 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.646589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:45233 (local address 127.17.188.65:51627)
0620 23:03:39.646760 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.646768 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:39.646843 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:39.646966 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:39.646967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:39.646969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:39.647027 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.647028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.647851 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.647852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.649144 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.649147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.649878 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.649879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.649961 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.649965 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:39.650015 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 23:03:39.650048 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":9,"thread_start_us":97,"threads_started":1}
I20260620 23:03:39.650215 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.646605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:35845 (local address 127.17.188.67:42205)
0620 23:03:39.646661 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.646663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.646673 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:03:39.646750 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.646751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.646760 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.646818 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:39.647001 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.647734 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.649231 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.649820 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.649889 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.649896 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.650112 (+ 216us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.650151 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:03:39.650434 22439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.646569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:42205 (local address 127.17.188.65:35845)
0620 23:03:39.646622 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.646635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:39.646677 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:39.646839 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:39.646840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:39.646842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:39.646917 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.646919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.647851 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.647852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.649144 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:39.649147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.649929 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.649930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:39.650021 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.650025 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:39.650093 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:03:39.650129 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":104}
I20260620 23:03:39.650520 22524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c315b5e980b949e48eb2b06eaa7174f8" candidate_uuid: "7369052f7cd94740a3db31cccdbffccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" is_pre_election: true
I20260620 23:03:39.650667 22524 raft_consensus.cc:2454] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7369052f7cd94740a3db31cccdbffccd in term 0.
I20260620 23:03:39.650725 22657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c315b5e980b949e48eb2b06eaa7174f8" candidate_uuid: "7369052f7cd94740a3db31cccdbffccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "472a40375d944986b5ec78b19b99b19a" is_pre_election: true
I20260620 23:03:39.650857 22657 raft_consensus.cc:2454] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7369052f7cd94740a3db31cccdbffccd in term 0.
I20260620 23:03:39.650933 22326 leader_election.cc:302] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7369052f7cd94740a3db31cccdbffccd, bc5d3c4d61d6497ea743f791d9642b8b; no voters:
I20260620 23:03:39.651052 22728 raft_consensus.cc:2781] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:39.651114 22728 raft_consensus.cc:482] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:39.651139 22728 raft_consensus.cc:3037] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:39.651762 22728 raft_consensus.cc:504] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } }
I20260620 23:03:39.651892 22728 leader_election.cc:288] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [CANDIDATE]: Term 1 election: Requested vote from peers 472a40375d944986b5ec78b19b99b19a (127.17.188.67:42205), bc5d3c4d61d6497ea743f791d9642b8b (127.17.188.66:45233)
I20260620 23:03:39.652035 22657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c315b5e980b949e48eb2b06eaa7174f8" candidate_uuid: "7369052f7cd94740a3db31cccdbffccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "472a40375d944986b5ec78b19b99b19a"
I20260620 23:03:39.652048 22524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c315b5e980b949e48eb2b06eaa7174f8" candidate_uuid: "7369052f7cd94740a3db31cccdbffccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5d3c4d61d6497ea743f791d9642b8b"
I20260620 23:03:39.652107 22657 raft_consensus.cc:3037] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:39.652107 22524 raft_consensus.cc:3037] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:39.652945 22524 raft_consensus.cc:2454] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7369052f7cd94740a3db31cccdbffccd in term 1.
I20260620 23:03:39.652945 22657 raft_consensus.cc:2454] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7369052f7cd94740a3db31cccdbffccd in term 1.
I20260620 23:03:39.653113 22324 leader_election.cc:302] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 472a40375d944986b5ec78b19b99b19a, 7369052f7cd94740a3db31cccdbffccd; no voters:
I20260620 23:03:39.653204 22728 raft_consensus.cc:2781] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:39.653406 22728 raft_consensus.cc:686] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 1 LEADER]: Becoming Leader. State: Replica: 7369052f7cd94740a3db31cccdbffccd, State: Running, Role: LEADER
I20260620 23:03:39.653527 22728 consensus_queue.cc:237] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } }
I20260620 23:03:39.654242 22247 catalog_manager.cc:5495] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd reported cstate change: term changed from 0 to 1, leader changed from <none> to 7369052f7cd94740a3db31cccdbffccd (127.17.188.65). New cstate: current_term: 1 leader_uuid: "7369052f7cd94740a3db31cccdbffccd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7369052f7cd94740a3db31cccdbffccd" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:39.665887 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.663492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48080 (local address 127.17.188.65:46345)
0620 23:03:39.663534 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.663536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.663544 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:39.663612 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.663613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.663634 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.663707 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:39.663848 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.664280 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.665074 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:39.665267 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:39.665390 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:39.665554 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:39.665634 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:39.665715 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.665756 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.665773 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:03:39.667333 22657 raft_consensus.cc:1261] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [term 1 FOLLOWER]: Refusing update from remote peer 7369052f7cd94740a3db31cccdbffccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:39.667354 22524 raft_consensus.cc:1261] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [term 1 FOLLOWER]: Refusing update from remote peer 7369052f7cd94740a3db31cccdbffccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:39.667629 22730 consensus_queue.cc:1035] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5d3c4d61d6497ea743f791d9642b8b" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:39.667752 22728 consensus_queue.cc:1035] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "472a40375d944986b5ec78b19b99b19a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:39.669986 22733 mvcc.cc:205] Tried to move back new op lower bound from 7299058154155163648 to 7299058154101272576. Current Snapshot: MvccSnapshot[applied={T|T < 7299058154155163648}]
I20260620 23:03:39.670601 22734 mvcc.cc:205] Tried to move back new op lower bound from 7299058154155163648 to 7299058154101272576. Current Snapshot: MvccSnapshot[applied={T|T < 7299058154155163648 or (T in {7299058154155163648})}]
I20260620 23:03:39.672510 22247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:39.672595 22247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:39.673642 22247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7028b8354b504a14a4123796fcf5e11d: Sending DeleteTablet for 3 replicas of tablet c315b5e980b949e48eb2b06eaa7174f8
I20260620 23:03:39.674077 22504 tablet_service.cc:1514] Processing DeleteTablet for tablet c315b5e980b949e48eb2b06eaa7174f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:39 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:52586
I20260620 23:03:39.674154 22371 tablet_service.cc:1514] Processing DeleteTablet for tablet c315b5e980b949e48eb2b06eaa7174f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:39 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:48078
I20260620 23:03:39.674177 22637 tablet_service.cc:1514] Processing DeleteTablet for tablet c315b5e980b949e48eb2b06eaa7174f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:39 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:41722
I20260620 23:03:39.674372 22744 tablet_replica.cc:330] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd: stopping tablet replica
I20260620 23:03:39.674465 22744 raft_consensus.cc:2229] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:39.674489 22743 tablet_replica.cc:330] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: stopping tablet replica
I20260620 23:03:39.674566 22744 raft_consensus.cc:2258] T c315b5e980b949e48eb2b06eaa7174f8 P 7369052f7cd94740a3db31cccdbffccd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:39.674580 22743 raft_consensus.cc:2229] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:39.674676 22743 raft_consensus.cc:2258] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:39.674698 22745 tablet_replica.cc:330] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: stopping tablet replica
I20260620 23:03:39.674769 22745 raft_consensus.cc:2229] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:39.674784 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 22309
I20260620 23:03:39.674849 22745 raft_consensus.cc:2258] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:39.675539 22745 ts_tablet_manager.cc:1905] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:39.676739 22745 ts_tablet_manager.cc:1918] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:39.676800 22745 log.cc:1196] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/c315b5e980b949e48eb2b06eaa7174f8
I20260620 23:03:39.676978 22745 ts_tablet_manager.cc:1939] T c315b5e980b949e48eb2b06eaa7174f8 P 472a40375d944986b5ec78b19b99b19a: Deleting consensus metadata
I20260620 23:03:39.677380 22231 catalog_manager.cc:4841] TS 472a40375d944986b5ec78b19b99b19a (127.17.188.67:42205): tablet c315b5e980b949e48eb2b06eaa7174f8 (table test-table [id=fccb725481964c20a01afe5ded860fe9]) successfully deleted
I20260620 23:03:39.677624 22743 ts_tablet_manager.cc:1905] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:39.679451 22743 ts_tablet_manager.cc:1918] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:39.679522 22743 log.cc:1196] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/c315b5e980b949e48eb2b06eaa7174f8
I20260620 23:03:39.679762 22743 ts_tablet_manager.cc:1939] T c315b5e980b949e48eb2b06eaa7174f8 P bc5d3c4d61d6497ea743f791d9642b8b: Deleting consensus metadata
I20260620 23:03:39.680143 22233 catalog_manager.cc:4841] TS bc5d3c4d61d6497ea743f791d9642b8b (127.17.188.66:45233): tablet c315b5e980b949e48eb2b06eaa7174f8 (table test-table [id=fccb725481964c20a01afe5ded860fe9]) successfully deleted
W20260620 23:03:39.681022 22233 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:46345 (error 108)
I20260620 23:03:39.681203 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 22442
I20260620 23:03:39.682199 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.682077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:46345 (local address 127.0.0.1:48090)
0620 23:03:39.682136 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.682178 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:46345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 23:03:39.682343 22233 catalog_manager.cc:4584] TS 7369052f7cd94740a3db31cccdbffccd (127.17.188.65:46345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c315b5e980b949e48eb2b06eaa7174f8: Network error: Client connection negotiation failed: client connection to 127.17.188.65:46345: connect: Connection refused (error 111)
I20260620 23:03:39.685905 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 22575
I20260620 23:03:39.690560 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 22216
2026-06-20T23:03:39Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1867 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T23:03:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:39Z Disabled control of system clock
I20260620 23:03:39.716954 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:36709
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:42545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:36709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 23:03:39.792940 22750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:39.793120 22750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:39.793141 22750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:39.794257 22750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:39.794296 22750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:39.794308 22750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:39.794322 22750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:39.794333 22750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:39.795527 22750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:36709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:36709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:39.795711 22750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:39.795943 22750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:03:39.800447 22750 server_base.cc:1034] running on GCE node
W20260620 23:03:39.800515 22756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:39.800421 22758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:39.800422 22755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.800840 22750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:39.801117 22750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:39.802263 22750 hybrid_clock.cc:648] HybridClock initialized: now 1781996619802258 us; error 34 us; skew 500 ppm
I20260620 23:03:39.803421 22750 webserver.cc:460] Webserver started at http://127.17.188.126:43107/ using document root <none> and password file <none>
I20260620 23:03:39.803694 22750 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:39.803757 22750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:39.803907 22750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:39.804849 22750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "7cc8735b55364594a0bf0cee0c3c89bb"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "02265157f6e3a3d28020ce26c2171706"
server_key_iv: "bab3b68e1f1fc926b5e7f8797e114ecd"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.805164 22750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "7cc8735b55364594a0bf0cee0c3c89bb"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "02265157f6e3a3d28020ce26c2171706"
server_key_iv: "bab3b68e1f1fc926b5e7f8797e114ecd"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.806497 22750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:39.807153 22764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.807327 22750 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:39.807423 22750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "7cc8735b55364594a0bf0cee0c3c89bb"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "02265157f6e3a3d28020ce26c2171706"
server_key_iv: "bab3b68e1f1fc926b5e7f8797e114ecd"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.807490 22750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:39.810736 22750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:39.810968 22750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:39.811103 22750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:39.814746 22750 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:36709
I20260620 23:03:39.814778 22816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:36709 every 8 connection(s)
I20260620 23:03:39.815079 22750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:39.815719 22817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:39.818235 22817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb: Bootstrap starting.
I20260620 23:03:39.818758 22817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:39.819051 22817 log.cc:824] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:39.819737 22817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb: No bootstrap required, opened a new log
I20260620 23:03:39.820442 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 22750
I20260620 23:03:39.820552 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:39.820708 18161 external_mini_cluster.cc:1428] Setting key 280c7b7ddcc989f8aa0ae40ce83d3d2c
I20260620 23:03:39.821892 22817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8735b55364594a0bf0cee0c3c89bb" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36709 } }
I20260620 23:03:39.822044 22817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:39.822079 22817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc8735b55364594a0bf0cee0c3c89bb, State: Initialized, Role: FOLLOWER
I20260620 23:03:39.822214 22817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8735b55364594a0bf0cee0c3c89bb" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36709 } }
I20260620 23:03:39.822326 22817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:39.822359 22817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:39.822405 22817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:39.823071 22817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8735b55364594a0bf0cee0c3c89bb" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36709 } }
I20260620 23:03:39.823182 22817 leader_election.cc:302] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cc8735b55364594a0bf0cee0c3c89bb; no voters:
I20260620 23:03:39.823213 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.821161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46800 (local address 127.17.188.126:36709)
0620 23:03:39.821341 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.821349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.821380 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:03:39.821854 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.821862 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.821909 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.822051 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:39.822739 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:39.822743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:39.822743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:39.822748 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:39.822789 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:39.822803 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:39.822879 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.822930 (+ 51us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.822983 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260620 23:03:39.823333 22817 leader_election.cc:288] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:39.823545 22817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:39.823593 22822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:39.823714 22822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [term 1 LEADER]: Becoming Leader. State: Replica: 7cc8735b55364594a0bf0cee0c3c89bb, State: Running, Role: LEADER
I20260620 23:03:39.823792 22822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8735b55364594a0bf0cee0c3c89bb" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36709 } }
I20260620 23:03:39.824137 22823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cc8735b55364594a0bf0cee0c3c89bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8735b55364594a0bf0cee0c3c89bb" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36709 } } }
I20260620 23:03:39.824225 22823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:39.824128 22822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cc8735b55364594a0bf0cee0c3c89bb. Latest consensus state: current_term: 1 leader_uuid: "7cc8735b55364594a0bf0cee0c3c89bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8735b55364594a0bf0cee0c3c89bb" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36709 } } }
I20260620 23:03:39.824407 22822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:39.824714 22831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:39.825163 22831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:39.826541 22831 catalog_manager.cc:1347] Generated new cluster ID: 0bc79d0074ac4fafbfd3b7486b2dc7cf
I20260620 23:03:39.826587 22831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:39.869921 22831 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:39.870069 22831 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:39.873982 22831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb: Generated new TSK 0
I20260620 23:03:39.874126 22831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:03:39.877218 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36709
--builtin_ntp_servers=127.17.188.84:42545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 23:03:39.955745 22841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:39.955942 22841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:39.955964 22841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:39.957052 22841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:39.957082 22841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:39.957106 22841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:39.958361 22841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:39.958546 22841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:39.958760 22841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:39.963600 22846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:39.963603 22847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.963665 22841 server_base.cc:1034] running on GCE node
W20260620 23:03:39.963586 22849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:39.964002 22841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:39.964259 22841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:39.965384 22841 hybrid_clock.cc:648] HybridClock initialized: now 1781996619965363 us; error 37 us; skew 500 ppm
I20260620 23:03:39.966580 22841 webserver.cc:460] Webserver started at http://127.17.188.65:42469/ using document root <none> and password file <none>
I20260620 23:03:39.966801 22841 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:39.966866 22841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:39.967021 22841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:39.968031 22841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "f9927992e81a4572a664cb1ac02a706e"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "ea6fb6a4b5860cb7c54678ba0c43bc47"
server_key_iv: "6282277b74b5978ef61f59598f094eb1"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.968379 22841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "f9927992e81a4572a664cb1ac02a706e"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "ea6fb6a4b5860cb7c54678ba0c43bc47"
server_key_iv: "6282277b74b5978ef61f59598f094eb1"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.970002 22841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:39.970688 22855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.970854 22841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:39.970958 22841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "f9927992e81a4572a664cb1ac02a706e"
format_stamp: "Formatted at 2026-06-20 23:03:39 on dist-test-slave-zpss"
server_key: "ea6fb6a4b5860cb7c54678ba0c43bc47"
server_key_iv: "6282277b74b5978ef61f59598f094eb1"
server_key_version: "encryptionkey@0"
I20260620 23:03:39.971036 22841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:39.974185 22841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:39.974439 22841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:39.974570 22841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:39.974774 22841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:39.975060 22841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:39.975111 22841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.975148 22841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:39.975178 22841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:39.980609 22841 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:42521
I20260620 23:03:39.980643 22968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:42521 every 8 connection(s)
I20260620 23:03:39.980944 22841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:39.981484 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 22841
I20260620 23:03:39.981597 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:39.981797 18161 external_mini_cluster.cc:1428] Setting key c0459c8e9fac269def6c52902669966d
I20260620 23:03:39.982684 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36709
--builtin_ntp_servers=127.17.188.84:42545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260620 23:03:39.984086 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.981714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:59411 (local address 127.17.188.126:36709)
0620 23:03:39.981760 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:39.981762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:39.981989 (+ 227us) server_negotiation.cc:407] Connection header received
0620 23:03:39.982545 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:39.982546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:39.982552 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:39.982597 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:39.983829 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:39.983831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:39.983831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:39.983836 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:39.983854 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:39.983859 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:39.983917 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:39.983992 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:03:39.984007 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:03:39.984261 22971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.981664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36709 (local address 127.17.188.65:59411)
0620 23:03:39.981857 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.981894 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:39.982452 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:39.982661 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:39.982664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:39.982674 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:39.983671 (+ 997us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:39.983672 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:39.983697 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:39.983722 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:39.983916 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:39.983921 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:39.983924 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:39.984000 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:03:39.984068 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260620 23:03:39.984755 22969 heartbeater.cc:347] Connected to a master server at 127.17.188.126:36709
I20260620 23:03:39.984839 22969 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:39.984939 22969 heartbeater.cc:511] Master 127.17.188.126:36709 requested a full tablet report, sending...
I20260620 23:03:39.985450 22781 ts_manager.cc:194] Registered new tserver with Master: f9927992e81a4572a664cb1ac02a706e (127.17.188.65:42521)
W20260620 23:03:40.062901 22972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:40.063125 22972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:40.063174 22972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:40.064308 22972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:40.064380 22972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:40.064424 22972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:40.065800 22972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:40.066061 22972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:40.066357 22972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:40.071308 22980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:40.071336 22978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:40.071406 22972 server_base.cc:1034] running on GCE node
W20260620 23:03:40.071374 22977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:40.071655 22972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:40.071885 22972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:40.073019 22972 hybrid_clock.cc:648] HybridClock initialized: now 1781996620073017 us; error 37 us; skew 500 ppm
I20260620 23:03:40.074427 22972 webserver.cc:460] Webserver started at http://127.17.188.66:43969/ using document root <none> and password file <none>
I20260620 23:03:40.074668 22972 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:40.074710 22972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:40.074887 22972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:40.075901 22972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "8bb526b1689949eb809d280d721450b3"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "9781a2e18f8abcc162712cfa0bf394e2"
server_key_iv: "04419d3fff4ff7da56d3e3e6d4a0f1cb"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.076258 22972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "8bb526b1689949eb809d280d721450b3"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "9781a2e18f8abcc162712cfa0bf394e2"
server_key_iv: "04419d3fff4ff7da56d3e3e6d4a0f1cb"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.077605 22972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:40.078374 22986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:40.078564 22972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:40.078653 22972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "8bb526b1689949eb809d280d721450b3"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "9781a2e18f8abcc162712cfa0bf394e2"
server_key_iv: "04419d3fff4ff7da56d3e3e6d4a0f1cb"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.078724 22972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:40.083014 22972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:40.083257 22972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:40.083386 22972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:40.083595 22972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:40.083891 22972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:40.083921 22972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:40.083973 22972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:40.084002 22972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:40.089516 22972 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:38111
I20260620 23:03:40.089560 23099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:38111 every 8 connection(s)
I20260620 23:03:40.089862 22972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:40.092653 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.090530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:53253 (local address 127.17.188.126:36709)
0620 23:03:40.090592 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.090594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.090832 (+ 238us) server_negotiation.cc:407] Connection header received
0620 23:03:40.091301 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.091302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.091306 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.091368 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.092443 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.092444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.092444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.092448 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.092462 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.092467 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.092509 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.092588 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.092600 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:03:40.092864 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.090505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36709 (local address 127.17.188.66:53253)
0620 23:03:40.090690 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.090746 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:40.091217 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:40.091437 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:40.091440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:40.091447 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:40.092311 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:40.092312 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:40.092340 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:40.092366 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:40.092512 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:40.092514 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:40.092516 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:03:40.092582 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 23:03:40.092628 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260620 23:03:40.093389 23100 heartbeater.cc:347] Connected to a master server at 127.17.188.126:36709
I20260620 23:03:40.093482 23100 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:40.093639 23100 heartbeater.cc:511] Master 127.17.188.126:36709 requested a full tablet report, sending...
I20260620 23:03:40.094000 22781 ts_manager.cc:194] Registered new tserver with Master: 8bb526b1689949eb809d280d721450b3 (127.17.188.66:38111)
I20260620 23:03:40.097913 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 22972
I20260620 23:03:40.097986 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:40.098120 18161 external_mini_cluster.cc:1428] Setting key bdab88cba5a096eb485b06d021d9bec8
I20260620 23:03:40.098968 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36709
--builtin_ntp_servers=127.17.188.84:42545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 23:03:40.176050 23103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:40.176278 23103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:40.176316 23103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:40.177488 23103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:40.177551 23103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:40.177595 23103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:40.178926 23103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:40.179184 23103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:40.179466 23103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:40.184499 23111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:40.184521 23108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:40.184490 23109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:40.184695 23103 server_base.cc:1034] running on GCE node
I20260620 23:03:40.184900 23103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:40.185150 23103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:40.186285 23103 hybrid_clock.cc:648] HybridClock initialized: now 1781996620186271 us; error 30 us; skew 500 ppm
I20260620 23:03:40.187579 23103 webserver.cc:460] Webserver started at http://127.17.188.67:38361/ using document root <none> and password file <none>
I20260620 23:03:40.187809 23103 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:40.187888 23103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:40.188042 23103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:40.189059 23103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "d93bf58136904bec9cb174dae58344a1"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "a6de18bd2e5c1a717b6f01f74aae2691"
server_key_iv: "aaeeb86dcaa6094a0414b99cf31f4b7f"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.189456 23103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "d93bf58136904bec9cb174dae58344a1"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "a6de18bd2e5c1a717b6f01f74aae2691"
server_key_iv: "aaeeb86dcaa6094a0414b99cf31f4b7f"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.190748 23103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:40.191468 23117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:40.191630 23103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:40.191715 23103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "d93bf58136904bec9cb174dae58344a1"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "a6de18bd2e5c1a717b6f01f74aae2691"
server_key_iv: "aaeeb86dcaa6094a0414b99cf31f4b7f"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.191766 23103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:40.194948 23103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:40.195142 23103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:40.195227 23103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:40.195374 23103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:40.195595 23103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:40.195623 23103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:40.195642 23103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:40.195655 23103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:40.200968 23103 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:42279
I20260620 23:03:40.201035 23230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:42279 every 8 connection(s)
I20260620 23:03:40.201239 23103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:40.203101 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 23103
I20260620 23:03:40.203200 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:40.203423 18161 external_mini_cluster.cc:1428] Setting key 8cf432970476305b51452bdd60840cbb
I20260620 23:03:40.204224 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.201914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:35313 (local address 127.17.188.126:36709)
0620 23:03:40.201961 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.201964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.202211 (+ 247us) server_negotiation.cc:407] Connection header received
0620 23:03:40.202610 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.202611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.202616 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.202668 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.203955 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.203956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.203956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.203961 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.203977 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.203981 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.204036 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.204144 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.204158 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:40.204440 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.201914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36709 (local address 127.17.188.67:35313)
0620 23:03:40.202098 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.202127 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:40.202530 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:40.202729 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:40.202732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:40.202740 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:40.203771 (+ 1031us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:40.203772 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:40.203812 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:40.203868 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:40.204050 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:40.204054 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:40.204057 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:40.204153 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 23:03:40.204213 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260620 23:03:40.204823 23231 heartbeater.cc:347] Connected to a master server at 127.17.188.126:36709
I20260620 23:03:40.204890 23231 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:40.205025 23231 heartbeater.cc:511] Master 127.17.188.126:36709 requested a full tablet report, sending...
I20260620 23:03:40.205428 22781 ts_manager.cc:194] Registered new tserver with Master: d93bf58136904bec9cb174dae58344a1 (127.17.188.67:42279)
I20260620 23:03:40.215360 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:40.217702 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.216684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46808 (local address 127.17.188.126:36709)
0620 23:03:40.216768 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.216770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.216830 (+ 60us) server_negotiation.cc:407] Connection header received
0620 23:03:40.216874 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.216875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.216879 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.216926 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.217487 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.217488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.217488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.217491 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.217503 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.217507 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.217550 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.217592 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.217602 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 23:03:40.219079 22781 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:40.220556 22781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:40.224305 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.222614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:42521 (local address 127.0.0.1:41106)
0620 23:03:40.222783 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.222821 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:40.222873 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:40.223246 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:40.223251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:40.223257 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:40.223856 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:40.223856 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:40.223869 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:40.223905 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:40.224124 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:40.224128 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:40.224132 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:40.224218 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:03:40.224231 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"spinlock_wait_cycles":384,"thread_start_us":92,"threads_started":1}
I20260620 23:03:40.224514 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.222701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.17.188.65:42521)
0620 23:03:40.223097 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.223102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.223112 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:03:40.223137 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.223140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.223150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.223234 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.223974 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.223982 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.223982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.223986 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.224037 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.224067 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.224129 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.224416 (+ 287us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.224436 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":118,"threads_started":1}
I20260620 23:03:40.224596 23242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.222614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:38111 (local address 127.0.0.1:44490)
0620 23:03:40.222981 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.222992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:40.223044 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:40.223497 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:40.223498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:40.223500 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:40.224190 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:40.224190 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:40.224196 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:40.224209 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:40.224455 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:40.224456 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:40.224457 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:03:40.224504 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 23:03:40.224516 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260620 23:03:40.224607 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.222819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.17.188.66:38111)
0620 23:03:40.223159 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.223162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.223170 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:40.223190 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.223192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.223198 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.223284 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.224287 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.224294 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.224294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.224298 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.224329 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.224374 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.224430 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.224513 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.224528 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":88,"threads_started":1}
I20260620 23:03:40.225168 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.222813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:42279 (local address 127.0.0.1:36722)
0620 23:03:40.223014 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.223022 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:40.223057 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:40.223866 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:40.223867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:40.223870 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:40.224256 (+ 386us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:40.224257 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:40.224263 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:40.224275 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:40.225034 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:40.225035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:40.225036 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:03:40.225091 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 23:03:40.225104 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":90,"threads_started":1}
I20260620 23:03:40.225168 22903 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef732684e4e4d09b36b86373c5697ad (DEFAULT_TABLE table=test-table [id=354fa87cbe5a40d3b34b2b9c0b36903b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:40.225441 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.223012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.17.188.67:42279)
0620 23:03:40.223488 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.223491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.223502 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:03:40.223526 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.223528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.223536 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.223606 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.224888 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.224897 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.224897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.224901 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.224933 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.224958 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.225025 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.225298 (+ 273us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.225346 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":83,"threads_started":1}
I20260620 23:03:40.225299 23034 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef732684e4e4d09b36b86373c5697ad (DEFAULT_TABLE table=test-table [id=354fa87cbe5a40d3b34b2b9c0b36903b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:40.225575 22903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ef732684e4e4d09b36b86373c5697ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:40.225680 23034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ef732684e4e4d09b36b86373c5697ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:40.228108 23245 tablet_bootstrap.cc:492] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e: Bootstrap starting.
I20260620 23:03:40.228197 23246 tablet_bootstrap.cc:492] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3: Bootstrap starting.
I20260620 23:03:40.228744 23246 tablet_bootstrap.cc:654] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:40.228790 23245 tablet_bootstrap.cc:654] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:40.228791 23165 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef732684e4e4d09b36b86373c5697ad (DEFAULT_TABLE table=test-table [id=354fa87cbe5a40d3b34b2b9c0b36903b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:40.229091 23246 log.cc:824] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:40.229120 23165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ef732684e4e4d09b36b86373c5697ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:40.229105 23245 log.cc:824] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:40.229835 23246 tablet_bootstrap.cc:492] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3: No bootstrap required, opened a new log
I20260620 23:03:40.229905 23246 ts_tablet_manager.cc:1397] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:40.232242 23249 tablet_bootstrap.cc:492] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: Bootstrap starting.
I20260620 23:03:40.232450 23246 raft_consensus.cc:348] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } }
I20260620 23:03:40.232622 23246 raft_consensus.cc:374] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:40.232669 23246 raft_consensus.cc:729] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bb526b1689949eb809d280d721450b3, State: Initialized, Role: FOLLOWER
I20260620 23:03:40.232779 23249 tablet_bootstrap.cc:654] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:40.232796 23246 consensus_queue.cc:260] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } }
I20260620 23:03:40.232875 23245 tablet_bootstrap.cc:492] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e: No bootstrap required, opened a new log
I20260620 23:03:40.232950 23245 ts_tablet_manager.cc:1397] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260620 23:03:40.233038 23246 ts_tablet_manager.cc:1428] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:40.233100 23249 log.cc:824] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:40.233148 23100 heartbeater.cc:503] Master 127.17.188.126:36709 was elected leader, sending a full tablet report...
I20260620 23:03:40.234097 23249 tablet_bootstrap.cc:492] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: No bootstrap required, opened a new log
I20260620 23:03:40.234172 23249 ts_tablet_manager.cc:1397] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:40.235055 23245 raft_consensus.cc:348] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } }
I20260620 23:03:40.235167 23245 raft_consensus.cc:374] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:40.235185 23245 raft_consensus.cc:729] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9927992e81a4572a664cb1ac02a706e, State: Initialized, Role: FOLLOWER
I20260620 23:03:40.235285 23245 consensus_queue.cc:260] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } }
I20260620 23:03:40.235523 23245 ts_tablet_manager.cc:1428] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:40.235559 22969 heartbeater.cc:503] Master 127.17.188.126:36709 was elected leader, sending a full tablet report...
I20260620 23:03:40.236227 23249 raft_consensus.cc:348] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } }
I20260620 23:03:40.236382 23249 raft_consensus.cc:374] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:40.236423 23249 raft_consensus.cc:729] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d93bf58136904bec9cb174dae58344a1, State: Initialized, Role: FOLLOWER
I20260620 23:03:40.236531 23249 consensus_queue.cc:260] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } }
I20260620 23:03:40.236755 23249 ts_tablet_manager.cc:1428] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:40.236783 23231 heartbeater.cc:503] Master 127.17.188.126:36709 was elected leader, sending a full tablet report...
W20260620 23:03:40.340372 23101 tablet.cc:2369] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:03:40.451884 23232 tablet.cc:2369] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:40.463289 23252 raft_consensus.cc:482] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:40.463392 23252 raft_consensus.cc:504] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } }
I20260620 23:03:40.463770 23252 leader_election.cc:288] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bb526b1689949eb809d280d721450b3 (127.17.188.66:38111), d93bf58136904bec9cb174dae58344a1 (127.17.188.67:42279)
I20260620 23:03:40.465045 22971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.463972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:38111 (local address 127.17.188.65:57939)
0620 23:03:40.464019 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.464031 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:40.464083 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:40.464261 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:40.464262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:40.464264 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:40.464686 (+ 422us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:40.464687 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:40.464693 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:40.464702 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:40.464892 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:40.464892 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:40.464894 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:03:40.464939 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 23:03:40.464950 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 23:03:40.465067 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.464065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:57939 (local address 127.17.188.66:38111)
0620 23:03:40.464129 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.464130 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.464141 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:03:40.464168 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.464169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.464174 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.464256 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.464804 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.464805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.464806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.464811 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.464832 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.464837 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.464880 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.464953 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.464968 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:03:40.465173 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.464218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:57105 (local address 127.17.188.67:42279)
0620 23:03:40.464299 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.464302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.464329 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:03:40.464416 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.464417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.464422 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.464469 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.464938 (+ 469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.464939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.464940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.464944 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.464959 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.464963 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.465035 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.465094 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.465109 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:03:40.465176 23254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.464095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:42279 (local address 127.17.188.65:57105)
0620 23:03:40.464258 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.464268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:40.464336 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:40.464485 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:40.464486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:40.464488 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:40.464824 (+ 336us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:40.464825 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:40.464829 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:40.464836 (+ 7us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:40.465041 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:40.465042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:40.465042 (+ 0us) client_negotiation.cc:769] Sending connection context
0620 23:03:40.465086 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:03:40.465098 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260620 23:03:40.465386 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef732684e4e4d09b36b86373c5697ad" candidate_uuid: "f9927992e81a4572a664cb1ac02a706e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bb526b1689949eb809d280d721450b3" is_pre_election: true
I20260620 23:03:40.465519 23054 raft_consensus.cc:2454] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9927992e81a4572a664cb1ac02a706e in term 0.
I20260620 23:03:40.465776 22859 leader_election.cc:302] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bb526b1689949eb809d280d721450b3, f9927992e81a4572a664cb1ac02a706e; no voters:
I20260620 23:03:40.465901 23252 raft_consensus.cc:2781] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:40.465952 23252 raft_consensus.cc:482] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:40.465912 23185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef732684e4e4d09b36b86373c5697ad" candidate_uuid: "f9927992e81a4572a664cb1ac02a706e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d93bf58136904bec9cb174dae58344a1" is_pre_election: true
I20260620 23:03:40.465997 23252 raft_consensus.cc:3037] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:40.466056 23185 raft_consensus.cc:2454] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9927992e81a4572a664cb1ac02a706e in term 0.
I20260620 23:03:40.466745 23252 raft_consensus.cc:504] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } }
I20260620 23:03:40.466866 23252 leader_election.cc:288] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [CANDIDATE]: Term 1 election: Requested vote from peers 8bb526b1689949eb809d280d721450b3 (127.17.188.66:38111), d93bf58136904bec9cb174dae58344a1 (127.17.188.67:42279)
I20260620 23:03:40.466998 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef732684e4e4d09b36b86373c5697ad" candidate_uuid: "f9927992e81a4572a664cb1ac02a706e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bb526b1689949eb809d280d721450b3"
I20260620 23:03:40.467038 23185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef732684e4e4d09b36b86373c5697ad" candidate_uuid: "f9927992e81a4572a664cb1ac02a706e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d93bf58136904bec9cb174dae58344a1"
I20260620 23:03:40.467072 23054 raft_consensus.cc:3037] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:40.467120 23185 raft_consensus.cc:3037] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:40.467734 23054 raft_consensus.cc:2454] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9927992e81a4572a664cb1ac02a706e in term 1.
I20260620 23:03:40.467890 22859 leader_election.cc:302] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bb526b1689949eb809d280d721450b3, f9927992e81a4572a664cb1ac02a706e; no voters:
I20260620 23:03:40.467931 23185 raft_consensus.cc:2454] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9927992e81a4572a664cb1ac02a706e in term 1.
I20260620 23:03:40.467958 23252 raft_consensus.cc:2781] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:40.468065 23252 raft_consensus.cc:686] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [term 1 LEADER]: Becoming Leader. State: Replica: f9927992e81a4572a664cb1ac02a706e, State: Running, Role: LEADER
I20260620 23:03:40.468145 23252 consensus_queue.cc:237] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } }
I20260620 23:03:40.468924 22780 catalog_manager.cc:5495] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e reported cstate change: term changed from 0 to 1, leader changed from <none> to f9927992e81a4572a664cb1ac02a706e (127.17.188.65). New cstate: current_term: 1 leader_uuid: "f9927992e81a4572a664cb1ac02a706e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9927992e81a4572a664cb1ac02a706e" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42521 } health_report { overall_health: HEALTHY } } }
W20260620 23:03:40.481496 22970 tablet.cc:2369] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:40.505915 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.504816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.17.188.65:42521)
0620 23:03:40.504884 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.504886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.504895 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:03:40.504939 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.504940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.504955 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.505028 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.505655 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.505656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.505656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.505660 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.505672 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.505676 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.505721 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.505782 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.505792 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:03:40.507565 23185 raft_consensus.cc:1261] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [term 1 FOLLOWER]: Refusing update from remote peer f9927992e81a4572a664cb1ac02a706e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:40.507597 23054 raft_consensus.cc:1261] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [term 1 FOLLOWER]: Refusing update from remote peer f9927992e81a4572a664cb1ac02a706e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:40.507923 23255 consensus_queue.cc:1035] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d93bf58136904bec9cb174dae58344a1" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:40.508036 23252 consensus_queue.cc:1035] T 3ef732684e4e4d09b36b86373c5697ad P f9927992e81a4572a664cb1ac02a706e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bb526b1689949eb809d280d721450b3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 38111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:40.511250 23260 mvcc.cc:205] Tried to move back new op lower bound from 7299058157596504064 to 7299058157438074880. Current Snapshot: MvccSnapshot[applied={T|T < 7299058157596504064 or (T in {7299058157596504064})}]
I20260620 23:03:40.511363 23259 mvcc.cc:205] Tried to move back new op lower bound from 7299058157596504064 to 7299058157438074880. Current Snapshot: MvccSnapshot[applied={T|T < 7299058157596504064 or (T in {7299058157596504064})}]
I20260620 23:03:40.512967 22781 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:40.513049 22781 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:40.513778 22781 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cc8735b55364594a0bf0cee0c3c89bb: Sending DeleteTablet for 3 replicas of tablet 3ef732684e4e4d09b36b86373c5697ad
I20260620 23:03:40.514070 22903 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ef732684e4e4d09b36b86373c5697ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:40 UTC) from {username='slave'} at 127.0.0.1:41106
I20260620 23:03:40.514108 23034 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ef732684e4e4d09b36b86373c5697ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:40 UTC) from {username='slave'} at 127.0.0.1:44490
I20260620 23:03:40.514303 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 22841
I20260620 23:03:40.514401 23165 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ef732684e4e4d09b36b86373c5697ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:40 UTC) from {username='slave'} at 127.0.0.1:36722
I20260620 23:03:40.514883 23270 tablet_replica.cc:330] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: stopping tablet replica
I20260620 23:03:40.514986 23270 raft_consensus.cc:2229] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:40.515105 23270 raft_consensus.cc:2258] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:40.515534 23270 ts_tablet_manager.cc:1905] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:40.517438 23269 tablet_replica.cc:330] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3: stopping tablet replica
I20260620 23:03:40.517557 23269 raft_consensus.cc:2229] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:40.517660 23269 raft_consensus.cc:2258] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:40.518040 23269 ts_tablet_manager.cc:1905] T 3ef732684e4e4d09b36b86373c5697ad P 8bb526b1689949eb809d280d721450b3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 23:03:40.519822 22765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:42521 (error 108)
I20260620 23:03:40.520071 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 22972
I20260620 23:03:40.521378 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.520143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:42521 (local address 127.0.0.1:41124)
0620 23:03:40.521264 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.521339 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1099}
W20260620 23:03:40.521919 22765 catalog_manager.cc:4584] TS f9927992e81a4572a664cb1ac02a706e (127.17.188.65:42521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ef732684e4e4d09b36b86373c5697ad: Network error: Client connection negotiation failed: client connection to 127.17.188.65:42521: connect: Connection refused (error 111)
I20260620 23:03:40.528738 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 23103
I20260620 23:03:40.528767 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.528646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:38111 (local address 127.0.0.1:44502)
0620 23:03:40.528701 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.528733 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.66:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 23:03:40.528851 23270 ts_tablet_manager.cc:1918] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:40.528932 23270 log.cc:1196] T 3ef732684e4e4d09b36b86373c5697ad P d93bf58136904bec9cb174dae58344a1: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/3ef732684e4e4d09b36b86373c5697ad
I20260620 23:03:40.533234 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 22750
I20260620 23:03:40.533392 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.533207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:42279 (local address 127.0.0.1:36734)
0620 23:03:40.533268 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.533321 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.67:42279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
2026-06-20T23:03:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (842 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 23:03:40.552531 18161 mini_oidc.cc:152] Starting JWKS server
I20260620 23:03:40.553759 18161 webserver.cc:460] Webserver started at https://127.0.0.1:43601/ using document root <none> and password file <none>
I20260620 23:03:40.553838 18161 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 23:03:40.554026 18161 webserver.cc:460] Webserver started at http://127.0.0.1:37335/ using document root <none> and password file <none>
2026-06-20T23:03:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:40Z Disabled control of system clock
I20260620 23:03:40.561535 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:33495
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:33495
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43601/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/testchainca.cert with env {}
W20260620 23:03:40.639863 23277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:40.640040 23277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:40.640060 23277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:40.641201 23277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:40.641242 23277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:40.641254 23277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:40.641304 23277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:40.641340 23277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 23:03:40.641360 23277 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 23:03:40.641398 23277 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43601/jwks.json
I20260620 23:03:40.642880 23277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:33495
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:33495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43601/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:40.643054 23277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:40.643195 23277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:40.647644 23282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:40.647621 23285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:40.647797 23277 server_base.cc:1034] running on GCE node
W20260620 23:03:40.647632 23283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:40.648137 23277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:40.648461 23277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:40.649598 23277 hybrid_clock.cc:648] HybridClock initialized: now 1781996620649562 us; error 52 us; skew 500 ppm
I20260620 23:03:40.650738 23277 webserver.cc:460] Webserver started at http://127.17.188.126:42635/ using document root <none> and password file <none>
I20260620 23:03:40.650969 23277 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:40.651012 23277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:40.651167 23277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:40.652148 23277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "99d86b0d3fe24e57956b3d2fb42dc276"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "cc6b69a9b4385c11e5680f3650a82750"
server_key_iv: "f18c1e1d96a3fb3bbce5af8bbf5b2fdc"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.652522 23277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "99d86b0d3fe24e57956b3d2fb42dc276"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "cc6b69a9b4385c11e5680f3650a82750"
server_key_iv: "f18c1e1d96a3fb3bbce5af8bbf5b2fdc"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.653833 23277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:40.654497 23291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:40.655376 23277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:40.655500 23277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "99d86b0d3fe24e57956b3d2fb42dc276"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "cc6b69a9b4385c11e5680f3650a82750"
server_key_iv: "f18c1e1d96a3fb3bbce5af8bbf5b2fdc"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.655627 23277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:40.665072 23277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:40.665402 23277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:40.665544 23277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:40.669070 23277 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:33495
I20260620 23:03:40.669098 23343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:33495 every 8 connection(s)
I20260620 23:03:40.669494 23277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:40.670398 23344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:40.672465 23344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276: Bootstrap starting.
I20260620 23:03:40.673010 23344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:40.673296 23344 log.cc:824] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:40.674043 23344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276: No bootstrap required, opened a new log
I20260620 23:03:40.675347 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 23277
I20260620 23:03:40.675454 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:40.675654 18161 external_mini_cluster.cc:1428] Setting key e64143839e12763bcf42251c7a820d7a
I20260620 23:03:40.675920 23344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d86b0d3fe24e57956b3d2fb42dc276" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33495 } }
I20260620 23:03:40.676074 23344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:40.676115 23344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d86b0d3fe24e57956b3d2fb42dc276, State: Initialized, Role: FOLLOWER
I20260620 23:03:40.676225 23344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d86b0d3fe24e57956b3d2fb42dc276" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33495 } }
I20260620 23:03:40.676343 23344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:40.676389 23344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:40.676458 23344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:40.680716 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.676147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.17.188.126:33495)
0620 23:03:40.676345 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.676353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.676385 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:03:40.676514 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.676520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.676591 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.676743 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.676913 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:40.678113 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:40.678955 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:40.679380 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:40.679510 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:40.680108 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.680111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.680112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.680278 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.680308 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.680318 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.680395 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.680476 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.680564 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260620 23:03:40.681620 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:03:40.684876 23344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d86b0d3fe24e57956b3d2fb42dc276" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33495 } }
I20260620 23:03:40.684990 23344 leader_election.cc:302] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99d86b0d3fe24e57956b3d2fb42dc276; no voters:
I20260620 23:03:40.685154 23344 leader_election.cc:288] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:40.685345 23350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:40.685436 23344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:40.685456 23350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [term 1 LEADER]: Becoming Leader. State: Replica: 99d86b0d3fe24e57956b3d2fb42dc276, State: Running, Role: LEADER
I20260620 23:03:40.685529 23350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d86b0d3fe24e57956b3d2fb42dc276" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33495 } }
I20260620 23:03:40.685875 23350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99d86b0d3fe24e57956b3d2fb42dc276. Latest consensus state: current_term: 1 leader_uuid: "99d86b0d3fe24e57956b3d2fb42dc276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d86b0d3fe24e57956b3d2fb42dc276" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33495 } } }
I20260620 23:03:40.685972 23350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:40.685883 23351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99d86b0d3fe24e57956b3d2fb42dc276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d86b0d3fe24e57956b3d2fb42dc276" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33495 } } }
I20260620 23:03:40.686126 23351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276 [sys.catalog]: This master's current role is: LEADER
W20260620 23:03:40.687180 23364 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:03:40.687285 23364 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:03:40.687386 23358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:40.687592 23358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:40.690630 23358 catalog_manager.cc:1347] Generated new cluster ID: e885107f40c844ecbf8b6c8ae00e8355
I20260620 23:03:40.690683 23358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:40.708755 23358 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:40.709262 23358 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:40.714131 23358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99d86b0d3fe24e57956b3d2fb42dc276: Generated new TSK 0
I20260620 23:03:40.714290 23358 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 23:03:40.744431 23374 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 23:03:40.744570 23374 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 23:03:40.744779 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.721711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.17.188.126:33495)
0620 23:03:40.721809 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.721810 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.721857 (+ 47us) server_negotiation.cc:407] Connection header received
0620 23:03:40.721942 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.721943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.721953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.722003 (+ 50us) server_negotiation.cc:226] Negotiated authn=JWT
0620 23:03:40.722141 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:40.722604 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:40.723709 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:40.723900 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:40.723953 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:40.724304 (+ 351us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 23:03:40.744435 (+ 20131us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43601/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 23:03:40.744718 (+ 283us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48260: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43601/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":72}
W20260620 23:03:40.744834 23348 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48260: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43601/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260620 23:03:40.744988 23373 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:03:40.721618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33495 (local address 127.0.0.1:48260)
0620 23:03:40.721786 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.721797 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:40.721863 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:40.722018 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:40.722019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:40.722021 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 23:03:40.722085 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:40.722087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:40.722684 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:40.722685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:40.723624 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:40.723626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:40.723975 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:40.723976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:40.724041 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:40.724063 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 23:03:40.744562 (+ 20499us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43601/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 23:03:40.744658 (+ 96us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.188.126:33495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43601/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":91,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.188.126:33495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43601/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 23:03:40.745383 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 23277
2026-06-20T23:03:40Z chronyd exiting
I20260620 23:03:40.780246 18161 test_util.cc:175] -----------------------------------------------
I20260620 23:03:40.780308 18161 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996599092328-18161-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 23:03:40.781041 18161 mini_oidc.cc:152] Starting JWKS server
I20260620 23:03:40.781766 18161 webserver.cc:460] Webserver started at https://127.0.0.1:44671/ using document root <none> and password file <none>
I20260620 23:03:40.781805 18161 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 23:03:40.781970 18161 webserver.cc:460] Webserver started at http://127.0.0.1:39779/ using document root <none> and password file <none>
2026-06-20T23:03:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:40Z Disabled control of system clock
I20260620 23:03:40.789739 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42647
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:35575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42647
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44671/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/testchainca.cert with env {}
W20260620 23:03:40.867087 23380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:40.867267 23380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:40.867288 23380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:40.868441 23380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:40.868481 23380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:40.868494 23380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:40.868531 23380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:40.868557 23380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 23:03:40.868579 23380 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 23:03:40.868607 23380 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44671/jwks.json
I20260620 23:03:40.870123 23380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42647
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44671/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:40.870368 23380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:40.870571 23380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:40.875200 23385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:40.875269 23386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:40.875288 23388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:40.875427 23380 server_base.cc:1034] running on GCE node
I20260620 23:03:40.875653 23380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:40.875933 23380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:40.877100 23380 hybrid_clock.cc:648] HybridClock initialized: now 1781996620877084 us; error 28 us; skew 500 ppm
I20260620 23:03:40.878345 23380 webserver.cc:460] Webserver started at http://127.17.188.126:45567/ using document root <none> and password file <none>
I20260620 23:03:40.878579 23380 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:40.878647 23380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:40.878774 23380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:40.879782 23380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "259f31a6bf0b49749e55908bc776b8e1"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "d350b9a6ffcf10439fb41670ecef6608"
server_key_iv: "54f2571fa0efc3b91cad10cbeddb3ea8"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.880143 23380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "259f31a6bf0b49749e55908bc776b8e1"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "d350b9a6ffcf10439fb41670ecef6608"
server_key_iv: "54f2571fa0efc3b91cad10cbeddb3ea8"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.881484 23380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:40.882172 23394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:40.882323 23380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:40.882417 23380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "259f31a6bf0b49749e55908bc776b8e1"
format_stamp: "Formatted at 2026-06-20 23:03:40 on dist-test-slave-zpss"
server_key: "d350b9a6ffcf10439fb41670ecef6608"
server_key_iv: "54f2571fa0efc3b91cad10cbeddb3ea8"
server_key_version: "encryptionkey@0"
I20260620 23:03:40.882507 23380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:40.895721 23380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:40.895988 23380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:40.896127 23380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:40.899900 23380 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:42647
I20260620 23:03:40.899927 23446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:42647 every 8 connection(s)
I20260620 23:03:40.900242 23380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:40.900877 23447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:40.903430 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 23380
I20260620 23:03:40.903460 23447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1: Bootstrap starting.
I20260620 23:03:40.903538 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:40.903712 18161 external_mini_cluster.cc:1428] Setting key f97a938cd5e53a69b59e3c5ac6c54c22
I20260620 23:03:40.904130 23447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:40.904444 23447 log.cc:824] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:40.905151 23447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1: No bootstrap required, opened a new log
I20260620 23:03:40.907341 23447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259f31a6bf0b49749e55908bc776b8e1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42647 } }
I20260620 23:03:40.907459 23447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:40.907480 23447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259f31a6bf0b49749e55908bc776b8e1, State: Initialized, Role: FOLLOWER
I20260620 23:03:40.907603 23447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259f31a6bf0b49749e55908bc776b8e1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42647 } }
I20260620 23:03:40.907673 23447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:40.907698 23447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:40.907734 23447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:40.909487 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.904135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42818 (local address 127.17.188.126:42647)
0620 23:03:40.904375 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:40.904392 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:40.904426 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:03:40.904992 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:40.905000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:40.905067 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:40.905211 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:40.905476 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:40.906804 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:40.907556 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:40.908285 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:40.908382 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:40.908936 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:40.908940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:40.908940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:40.909092 (+ 152us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:40.909126 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:40.909136 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:40.909187 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:40.909242 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 23:03:40.909296 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260620 23:03:40.910296 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:03:40.994959 23447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259f31a6bf0b49749e55908bc776b8e1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42647 } }
I20260620 23:03:40.995151 23447 leader_election.cc:302] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 259f31a6bf0b49749e55908bc776b8e1; no voters:
I20260620 23:03:40.995355 23447 leader_election.cc:288] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:40.995433 23452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:40.995599 23452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [term 1 LEADER]: Becoming Leader. State: Replica: 259f31a6bf0b49749e55908bc776b8e1, State: Running, Role: LEADER
I20260620 23:03:40.995661 23447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:40.995708 23452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259f31a6bf0b49749e55908bc776b8e1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42647 } }
I20260620 23:03:40.996049 23452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 259f31a6bf0b49749e55908bc776b8e1. Latest consensus state: current_term: 1 leader_uuid: "259f31a6bf0b49749e55908bc776b8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259f31a6bf0b49749e55908bc776b8e1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42647 } } }
I20260620 23:03:40.996060 23453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "259f31a6bf0b49749e55908bc776b8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259f31a6bf0b49749e55908bc776b8e1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42647 } } }
I20260620 23:03:40.996136 23452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:40.996165 23453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:40.996680 23458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:40.997225 23458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:41.000540 23458 catalog_manager.cc:1347] Generated new cluster ID: 35fa00894eae454295b9ef04e0ec6ffb
I20260620 23:03:41.000617 23458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:41.016016 23458 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:41.016510 23458 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:41.019235 23458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 259f31a6bf0b49749e55908bc776b8e1: Generated new TSK 0
I20260620 23:03:41.019403 23458 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 23:03:41.042407 23477 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 23:03:41.042484 23477 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 23:03:41.042825 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:41.027947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.17.188.126:42647)
0620 23:03:41.028015 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:41.028017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:41.028119 (+ 102us) server_negotiation.cc:407] Connection header received
0620 23:03:41.028154 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:41.028155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:41.028168 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:41.028247 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0620 23:03:41.028363 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.029968 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.030700 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.031011 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.031082 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:41.031226 (+ 144us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 23:03:41.042614 (+ 11388us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 23:03:41.042757 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 23:03:41.042886 23450 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260620 23:03:41.042863 23476 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:03:41.027861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42647 (local address 127.0.0.1:42830)
0620 23:03:41.028028 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:41.028039 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:41.028100 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:41.028251 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:41.028252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:41.028253 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0620 23:03:41.028297 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:41.028299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.030044 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.030045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:41.030623 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:41.030625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.031089 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.031090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:41.031131 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:41.031135 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 23:03:41.042712 (+ 11577us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 23:03:41.042800 (+ 88us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.188.126:42647: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260620 23:03:41.043183 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 23380
2026-06-20T23:03:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (299 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 23:03:41.080390 18161 mini_oidc.cc:152] Starting JWKS server
I20260620 23:03:41.081065 18161 webserver.cc:460] Webserver started at https://127.0.0.1:36919/ using document root <none> and password file <none>
I20260620 23:03:41.081105 18161 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 23:03:41.081279 18161 webserver.cc:460] Webserver started at http://127.0.0.1:35725/ using document root <none> and password file <none>
2026-06-20T23:03:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:41Z Disabled control of system clock
I20260620 23:03:41.089265 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:36307
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:45537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:36307
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36919/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 23:03:41.167913 23483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:41.168115 23483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:41.168138 23483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:41.169375 23483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:41.169420 23483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:41.169433 23483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:41.169448 23483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:41.169462 23483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 23:03:41.169476 23483 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 23:03:41.169497 23483 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36919/jwks.json
I20260620 23:03:41.170799 23483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:45537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:36307
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:36307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36919/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:41.170989 23483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:41.171221 23483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:41.175972 23489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:41.175990 23491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:41.175973 23488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:41.176182 23483 server_base.cc:1034] running on GCE node
I20260620 23:03:41.176419 23483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:41.176741 23483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:41.177906 23483 hybrid_clock.cc:648] HybridClock initialized: now 1781996621177884 us; error 56 us; skew 500 ppm
I20260620 23:03:41.179075 23483 webserver.cc:460] Webserver started at http://127.17.188.126:37595/ using document root <none> and password file <none>
I20260620 23:03:41.179319 23483 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:41.179387 23483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:41.179543 23483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:41.180555 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "d9755fae07ec44f2bc0077faab3e51de"
format_stamp: "Formatted at 2026-06-20 23:03:41 on dist-test-slave-zpss"
server_key: "9a0fa65299564ca23d826fd304a12168"
server_key_iv: "6ee12ebe9536b1b546880118276cd22f"
server_key_version: "encryptionkey@0"
I20260620 23:03:41.180929 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "d9755fae07ec44f2bc0077faab3e51de"
format_stamp: "Formatted at 2026-06-20 23:03:41 on dist-test-slave-zpss"
server_key: "9a0fa65299564ca23d826fd304a12168"
server_key_iv: "6ee12ebe9536b1b546880118276cd22f"
server_key_version: "encryptionkey@0"
I20260620 23:03:41.182531 23483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:03:41.183270 23497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:41.183439 23483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:41.183539 23483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "d9755fae07ec44f2bc0077faab3e51de"
format_stamp: "Formatted at 2026-06-20 23:03:41 on dist-test-slave-zpss"
server_key: "9a0fa65299564ca23d826fd304a12168"
server_key_iv: "6ee12ebe9536b1b546880118276cd22f"
server_key_version: "encryptionkey@0"
I20260620 23:03:41.183621 23483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:41.198761 23483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:41.199070 23483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:41.199222 23483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:41.204001 23483 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:36307
I20260620 23:03:41.204016 23549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:36307 every 8 connection(s)
I20260620 23:03:41.204420 23483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:41.205060 23550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:41.207732 23550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de: Bootstrap starting.
I20260620 23:03:41.208316 23550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:41.208675 23550 log.cc:824] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:41.209379 23550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de: No bootstrap required, opened a new log
I20260620 23:03:41.210917 23550 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9755fae07ec44f2bc0077faab3e51de" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36307 } }
I20260620 23:03:41.211071 23550 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:41.211117 23550 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9755fae07ec44f2bc0077faab3e51de, State: Initialized, Role: FOLLOWER
I20260620 23:03:41.211272 23550 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9755fae07ec44f2bc0077faab3e51de" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36307 } }
I20260620 23:03:41.211352 23550 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:41.211393 23550 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:41.211443 23550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:41.213177 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 23483
I20260620 23:03:41.213291 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:41.213470 18161 external_mini_cluster.cc:1428] Setting key b0258c78b37c668817a845f92e8b0b42
I20260620 23:03:41.218226 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:41.213940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46804 (local address 127.17.188.126:36307)
0620 23:03:41.214147 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:41.214156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:41.214186 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:03:41.214286 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:41.214291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:41.214341 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:41.214478 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:41.214670 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.215505 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.216272 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.216680 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.216811 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:41.217502 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:41.217506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:41.217506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:41.217687 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:41.217717 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:41.217727 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:41.217796 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:41.217866 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 23:03:41.217940 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260620 23:03:41.218993 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:03:41.304021 23550 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9755fae07ec44f2bc0077faab3e51de" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36307 } }
I20260620 23:03:41.304179 23550 leader_election.cc:302] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9755fae07ec44f2bc0077faab3e51de; no voters:
I20260620 23:03:41.304456 23550 leader_election.cc:288] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:41.304538 23555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:41.304708 23555 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [term 1 LEADER]: Becoming Leader. State: Replica: d9755fae07ec44f2bc0077faab3e51de, State: Running, Role: LEADER
I20260620 23:03:41.304741 23550 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:41.304819 23555 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9755fae07ec44f2bc0077faab3e51de" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36307 } }
I20260620 23:03:41.305251 23557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9755fae07ec44f2bc0077faab3e51de. Latest consensus state: current_term: 1 leader_uuid: "d9755fae07ec44f2bc0077faab3e51de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9755fae07ec44f2bc0077faab3e51de" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36307 } } }
I20260620 23:03:41.305287 23556 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9755fae07ec44f2bc0077faab3e51de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9755fae07ec44f2bc0077faab3e51de" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36307 } } }
I20260620 23:03:41.305496 23556 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:41.305467 23557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:41.305845 23564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:41.306360 23564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:41.308146 23564 catalog_manager.cc:1347] Generated new cluster ID: ccaede1369064f9bb3f2e21ad6ee7aaf
I20260620 23:03:41.308202 23564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:41.331483 23564 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:41.332155 23564 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:41.338230 23564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9755fae07ec44f2bc0077faab3e51de: Generated new TSK 0
I20260620 23:03:41.338423 23564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:03:41.359262 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:41.355444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46806 (local address 127.17.188.126:36307)
0620 23:03:41.355506 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:41.355508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:41.355637 (+ 129us) server_negotiation.cc:407] Connection header received
0620 23:03:41.355713 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:41.355713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:41.355725 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:41.355784 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0620 23:03:41.355944 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.356426 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.357659 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.357842 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.357893 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:41.358104 (+ 211us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 23:03:41.359053 (+ 949us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36919/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 23:03:41.359193 (+ 140us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46806: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36919/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":36}
W20260620 23:03:41.359328 23554 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46806: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36919/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 23:03:41.359715 23579 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:03:41.355386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36307 (local address 127.0.0.1:46806)
0620 23:03:41.355565 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:41.355576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:41.355643 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:41.355802 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:41.355803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:41.355805 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 23:03:41.355869 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:41.355872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.356534 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.356535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:41.357566 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:41.357569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.357920 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.357922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:41.357985 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:41.357990 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 23:03:41.359152 (+ 1162us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36919/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 23:03:41.359260 (+ 108us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.188.126:36307: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36919/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":117,"threads_started":1}
I20260620 23:03:41.359925 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 23483
E20260620 23:03:41.369776 23580 webserver.cc:533] Webserver: sslize failed:
E20260620 23:03:41.369850 23580 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-20T23:03:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (317 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T23:03:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:41Z Disabled control of system clock
I20260620 23:03:41.404714 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:41233
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:41233
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/testchainca.cert with env {}
W20260620 23:03:41.485524 23584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:41.485705 23584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:41.485726 23584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:41.486745 23584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:41.486779 23584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:41.486799 23584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:41.486810 23584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:41.486822 23584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 23:03:41.486833 23584 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 23:03:41.486847 23584 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 23:03:41.488229 23584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:41233
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:41233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:41.488469 23584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:41.488688 23584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:41.493647 23592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:41.493669 23589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:41.493822 23584 server_base.cc:1034] running on GCE node
W20260620 23:03:41.493669 23590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:41.494112 23584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:41.494424 23584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:41.495587 23584 hybrid_clock.cc:648] HybridClock initialized: now 1781996621495579 us; error 40 us; skew 500 ppm
I20260620 23:03:41.496759 23584 webserver.cc:460] Webserver started at http://127.17.188.126:39777/ using document root <none> and password file <none>
I20260620 23:03:41.496991 23584 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:41.497058 23584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:41.497189 23584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:41.498241 23584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "5de079ca3a5342e2875c769c8deaa6c7"
format_stamp: "Formatted at 2026-06-20 23:03:41 on dist-test-slave-zpss"
server_key: "8fedbf628a327ace583b24366a26b18a"
server_key_iv: "3983017614c879a29028c79322eac40f"
server_key_version: "encryptionkey@0"
I20260620 23:03:41.498606 23584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "5de079ca3a5342e2875c769c8deaa6c7"
format_stamp: "Formatted at 2026-06-20 23:03:41 on dist-test-slave-zpss"
server_key: "8fedbf628a327ace583b24366a26b18a"
server_key_iv: "3983017614c879a29028c79322eac40f"
server_key_version: "encryptionkey@0"
I20260620 23:03:41.499846 23584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:03:41.500545 23598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:41.500718 23584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:41.500811 23584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "5de079ca3a5342e2875c769c8deaa6c7"
format_stamp: "Formatted at 2026-06-20 23:03:41 on dist-test-slave-zpss"
server_key: "8fedbf628a327ace583b24366a26b18a"
server_key_iv: "3983017614c879a29028c79322eac40f"
server_key_version: "encryptionkey@0"
I20260620 23:03:41.500897 23584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:41.509063 23584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:41.509366 23584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:41.509497 23584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:41.513024 23584 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:41233
I20260620 23:03:41.513056 23650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:41233 every 8 connection(s)
I20260620 23:03:41.513455 23584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:41.514328 23651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:41.516453 23651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7: Bootstrap starting.
I20260620 23:03:41.517009 23651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:41.517287 23651 log.cc:824] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:41.518019 23651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7: No bootstrap required, opened a new log
I20260620 23:03:41.518512 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 23584
I20260620 23:03:41.518631 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:41.518790 18161 external_mini_cluster.cc:1428] Setting key a5c79548a01850e472110e1c400c9ba0
I20260620 23:03:41.520227 23651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de079ca3a5342e2875c769c8deaa6c7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41233 } }
I20260620 23:03:41.520391 23651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:41.520428 23651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de079ca3a5342e2875c769c8deaa6c7, State: Initialized, Role: FOLLOWER
I20260620 23:03:41.520562 23651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de079ca3a5342e2875c769c8deaa6c7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41233 } }
I20260620 23:03:41.520640 23651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:41.520670 23651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:41.520713 23651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:41.524266 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:41.519247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38360 (local address 127.17.188.126:41233)
0620 23:03:41.519525 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:41.519534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:41.519566 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:03:41.520195 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:41.520204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:41.520279 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:41.520427 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:41.520668 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.521988 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.522689 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.523074 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.523171 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:41.523693 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:41.523697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:41.523697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:41.523861 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:41.523890 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:41.523900 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:41.523951 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:41.524006 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 23:03:41.524095 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260620 23:03:41.525130 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:03:41.599710 23651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de079ca3a5342e2875c769c8deaa6c7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41233 } }
I20260620 23:03:41.599874 23651 leader_election.cc:302] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5de079ca3a5342e2875c769c8deaa6c7; no voters:
I20260620 23:03:41.600186 23651 leader_election.cc:288] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:41.600281 23656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:41.600445 23656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [term 1 LEADER]: Becoming Leader. State: Replica: 5de079ca3a5342e2875c769c8deaa6c7, State: Running, Role: LEADER
I20260620 23:03:41.600478 23651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:41.600569 23656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de079ca3a5342e2875c769c8deaa6c7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41233 } }
I20260620 23:03:41.601015 23658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5de079ca3a5342e2875c769c8deaa6c7. Latest consensus state: current_term: 1 leader_uuid: "5de079ca3a5342e2875c769c8deaa6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de079ca3a5342e2875c769c8deaa6c7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41233 } } }
I20260620 23:03:41.601043 23657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5de079ca3a5342e2875c769c8deaa6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de079ca3a5342e2875c769c8deaa6c7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41233 } } }
I20260620 23:03:41.601214 23657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:41.601194 23658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:41.601900 23665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:41.602409 23665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:41.603994 23665 catalog_manager.cc:1347] Generated new cluster ID: 08ef98ee9ea74788a277dcced6700051
I20260620 23:03:41.604058 23665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:41.630846 23665 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:41.631346 23665 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:41.638340 23665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5de079ca3a5342e2875c769c8deaa6c7: Generated new TSK 0
I20260620 23:03:41.638520 23665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:03:41.676990 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:41.644706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.17.188.126:41233)
0620 23:03:41.644766 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:41.644768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:41.644871 (+ 103us) server_negotiation.cc:407] Connection header received
0620 23:03:41.644964 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:41.644965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:41.644975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:41.645034 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0620 23:03:41.645202 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.645701 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.646851 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.647040 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.647089 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:41.647282 (+ 193us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 23:03:41.676806 (+ 29524us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 23:03:41.676937 (+ 131us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38372: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":36}
W20260620 23:03:41.677070 23655 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38372: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 23:03:41.677192 23680 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:03:41.644641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:41233 (local address 127.0.0.1:38372)
0620 23:03:41.644812 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:41.644824 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:41.644890 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:41.645059 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:41.645060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:41.645062 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 23:03:41.645125 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:41.645128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.645820 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.645821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:41.646773 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:41.646776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:41.647121 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:41.647122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:41.647187 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:41.647192 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 23:03:41.676950 (+ 29758us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 23:03:41.677091 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.188.126:41233: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260620 23:03:41.677601 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 23584
2026-06-20T23:03:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (296 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:42 dist-test-slave-zpss krb5kdc[23692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:42 dist-test-slave-zpss krb5kdc[23692](info): set up 2 sockets
Jun 20 23:03:42 dist-test-slave-zpss krb5kdc[23692](info): commencing operation
krb5kdc: starting...
W20260620 23:03:43.325775 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.453s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:43 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:03:43.434369 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34387
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:34387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:43.514967 23705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:43.515196 23705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:43.515236 23705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:43.516376 23705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:43.516450 23705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:43.516476 23705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:43.516506 23705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:43.516534 23705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:43.518056 23705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:34387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:43.518316 23705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:43.518558 23705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:43.523393 23713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:43.523479 23711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:43.523511 23705 server_base.cc:1034] running on GCE node
W20260620 23:03:43.523488 23710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:43.523962 23705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:43.524189 23705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:43.525352 23705 hybrid_clock.cc:648] HybridClock initialized: now 1781996623525340 us; error 30 us; skew 500 ppm
Jun 20 23:03:43 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:43.527725 23705 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:03:43.528180 23705 webserver.cc:460] Webserver started at http://127.17.188.126:39793/ using document root <none> and password file <none>
I20260620 23:03:43.528394 23705 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:43.528442 23705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:43.528584 23705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:43.529654 23705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "1dea69ab142c4c45a293c81fe5b8803b"
format_stamp: "Formatted at 2026-06-20 23:03:43 on dist-test-slave-zpss"
server_key: "3556e0acbee228076e7d51d4a22b8bc0"
server_key_iv: "0c54414334e782f132a2a46d790d9970"
server_key_version: "encryptionkey@0"
I20260620 23:03:43.530021 23705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "1dea69ab142c4c45a293c81fe5b8803b"
format_stamp: "Formatted at 2026-06-20 23:03:43 on dist-test-slave-zpss"
server_key: "3556e0acbee228076e7d51d4a22b8bc0"
server_key_iv: "0c54414334e782f132a2a46d790d9970"
server_key_version: "encryptionkey@0"
I20260620 23:03:43.531347 23705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:03:43.532019 23720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:43.532193 23705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:43.532280 23705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "1dea69ab142c4c45a293c81fe5b8803b"
format_stamp: "Formatted at 2026-06-20 23:03:43 on dist-test-slave-zpss"
server_key: "3556e0acbee228076e7d51d4a22b8bc0"
server_key_iv: "0c54414334e782f132a2a46d790d9970"
server_key_version: "encryptionkey@0"
I20260620 23:03:43.532351 23705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:43.548210 23705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:43.549899 23705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:43.550050 23705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:43.553680 23705 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:34387
I20260620 23:03:43.553768 23772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:34387 every 8 connection(s)
I20260620 23:03:43.554042 23705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:43.554802 23773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:43.556768 23773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b: Bootstrap starting.
I20260620 23:03:43.557227 23773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:43.557488 23773 log.cc:824] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:43.558017 23773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b: No bootstrap required, opened a new log
I20260620 23:03:43.558347 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 23705
I20260620 23:03:43.558449 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:43.558610 18161 external_mini_cluster.cc:1428] Setting key 1f7cca8694c8022d44577bfe8801a1ea
I20260620 23:03:43.560000 23773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dea69ab142c4c45a293c81fe5b8803b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34387 } }
I20260620 23:03:43.560153 23773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:43.560192 23773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dea69ab142c4c45a293c81fe5b8803b, State: Initialized, Role: FOLLOWER
I20260620 23:03:43.560374 23773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dea69ab142c4c45a293c81fe5b8803b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34387 } }
I20260620 23:03:43.560465 23773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:43.560497 23773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:43.560549 23773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:43.561515 23773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dea69ab142c4c45a293c81fe5b8803b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34387 } }
I20260620 23:03:43.561604 23773 leader_election.cc:302] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dea69ab142c4c45a293c81fe5b8803b; no voters:
I20260620 23:03:43.561808 23773 leader_election.cc:288] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:43.561856 23778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:43.562012 23778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [term 1 LEADER]: Becoming Leader. State: Replica: 1dea69ab142c4c45a293c81fe5b8803b, State: Running, Role: LEADER
I20260620 23:03:43.562036 23773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:43.562145 23778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dea69ab142c4c45a293c81fe5b8803b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34387 } }
I20260620 23:03:43.563342 23780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dea69ab142c4c45a293c81fe5b8803b. Latest consensus state: current_term: 1 leader_uuid: "1dea69ab142c4c45a293c81fe5b8803b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dea69ab142c4c45a293c81fe5b8803b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34387 } } }
I20260620 23:03:43.563441 23780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [sys.catalog]: This master's current role is: LEADER
W20260620 23:03:43.563834 23793 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:03:43.564054 23793 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:03:43.564154 23794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:43.564332 23794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:43.564060 23779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dea69ab142c4c45a293c81fe5b8803b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dea69ab142c4c45a293c81fe5b8803b" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34387 } } }
I20260620 23:03:43.564467 23779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:43.565689 23794 catalog_manager.cc:1347] Generated new cluster ID: 4b333a30851b4bf493686118411903cd
I20260620 23:03:43.565735 23794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:03:43 dist-test-slave-zpss krb5kdc[23692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:43.569378 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:43.559073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.17.188.126:34387)
0620 23:03:43.559247 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:43.559254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:43.559275 (+ 21us) server_negotiation.cc:407] Connection header received
0620 23:03:43.559984 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:43.559990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:43.560033 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:43.560138 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:43.560941 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.562022 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.563105 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.563737 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.563844 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:43.566300 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:43.566304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:43.566305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:43.566314 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:43.567849 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:43.568215 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:43.568218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:43.568218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:43.568333 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:43.568569 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:43.568570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:43.568571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:43.568797 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:43.568971 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:43.569039 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 23:03:43.569172 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260620 23:03:43.575700 23794 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:43.576292 23794 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:43.579691 23794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dea69ab142c4c45a293c81fe5b8803b: Generated new TSK 0
I20260620 23:03:43.579839 23794 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:03:43.607372 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34387
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:43.705574 23799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:43.705757 23799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:43.705778 23799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:43.706900 23799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:43.706939 23799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:43.706976 23799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:43.708325 23799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:43.708531 23799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:43.708758 23799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:03:43.713928 23799 server_base.cc:1034] running on GCE node
W20260620 23:03:43.714022 23804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:43.714071 23805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:43.714015 23807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:43.714469 23799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:43.714726 23799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:43.715884 23799 hybrid_clock.cc:648] HybridClock initialized: now 1781996623715878 us; error 34 us; skew 500 ppm
Jun 20 23:03:43 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:43.718474 23799 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:03:43.718921 23799 webserver.cc:460] Webserver started at http://127.17.188.65:40889/ using document root <none> and password file <none>
I20260620 23:03:43.719133 23799 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:43.719203 23799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:43.719349 23799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:43.720464 23799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "85ebda1b0a9e4f55b3916bd087420ec2"
format_stamp: "Formatted at 2026-06-20 23:03:43 on dist-test-slave-zpss"
server_key: "4e56a25d00f04177b4c8f4ed33e3fed9"
server_key_iv: "b4e892e428e25fe5d44081cb11ec739b"
server_key_version: "encryptionkey@0"
I20260620 23:03:43.720836 23799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "85ebda1b0a9e4f55b3916bd087420ec2"
format_stamp: "Formatted at 2026-06-20 23:03:43 on dist-test-slave-zpss"
server_key: "4e56a25d00f04177b4c8f4ed33e3fed9"
server_key_iv: "b4e892e428e25fe5d44081cb11ec739b"
server_key_version: "encryptionkey@0"
I20260620 23:03:43.722307 23799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:43.723011 23814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:43.723170 23799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:43.723285 23799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "85ebda1b0a9e4f55b3916bd087420ec2"
format_stamp: "Formatted at 2026-06-20 23:03:43 on dist-test-slave-zpss"
server_key: "4e56a25d00f04177b4c8f4ed33e3fed9"
server_key_iv: "b4e892e428e25fe5d44081cb11ec739b"
server_key_version: "encryptionkey@0"
I20260620 23:03:43.723353 23799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:43.737197 23799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:43.739255 23799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:43.739414 23799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:43.739630 23799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:43.739914 23799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:43.739951 23799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:43.739971 23799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:43.740015 23799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:43.745430 23799 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:33411
I20260620 23:03:43.745469 23927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:33411 every 8 connection(s)
I20260620 23:03:43.745829 23799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:03:43 dist-test-slave-zpss krb5kdc[23692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:43.752355 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 23799
I20260620 23:03:43.752463 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:43.752688 18161 external_mini_cluster.cc:1428] Setting key 647c88772ada6b5d9ee2dec719c9d4f3
I20260620 23:03:43.754868 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:43.746477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:53703 (local address 127.17.188.126:34387)
0620 23:03:43.746538 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:43.746540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:43.746743 (+ 203us) server_negotiation.cc:407] Connection header received
0620 23:03:43.747653 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:43.747655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:43.747669 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:43.747728 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:43.748261 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.748706 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.749768 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.749984 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.750067 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:43.751944 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:43.751945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:43.751945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:43.751954 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:43.753548 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:43.753933 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:43.753934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:43.753935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:43.753997 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:43.754215 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:43.754216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:43.754216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:43.754351 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:43.754420 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:43.754643 (+ 223us) server_negotiation.cc:299] Negotiation successful
0620 23:03:43.754737 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:03:43.754994 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:43.746484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34387 (local address 127.17.188.65:53703)
0620 23:03:43.746637 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:43.746668 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:43.747529 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:43.747796 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:43.747799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:43.748017 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:43.748172 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:43.748175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.748789 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.748790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:43.749653 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:43.749655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.750078 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.750079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:43.750194 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:43.750556 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:43.750563 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:43.751866 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:43.753619 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:43.753624 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:43.753625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:43.753861 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:43.754062 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:43.754062 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:43.754063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:43.754153 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:43.754413 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:43.754417 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:43.754516 (+ 99us) client_negotiation.cc:769] Sending connection context
0620 23:03:43.754593 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 23:03:43.754737 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260620 23:03:43.755467 23928 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34387
I20260620 23:03:43.755582 23928 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:43.755741 23928 heartbeater.cc:511] Master 127.17.188.126:34387 requested a full tablet report, sending...
I20260620 23:03:43.756232 23737 ts_manager.cc:194] Registered new tserver with Master: 85ebda1b0a9e4f55b3916bd087420ec2 (127.17.188.65:33411)
I20260620 23:03:43.756824 23737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:53703
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:03:43.776002 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34387
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:43.855878 23933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:43.856061 23933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:43.856084 23933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:43.857095 23933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:43.857125 23933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:43.857148 23933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:43.858486 23933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:43.858697 23933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:43.858882 23933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:43.863729 23938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:43.863866 23933 server_base.cc:1034] running on GCE node
W20260620 23:03:43.863788 23939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:43.863721 23941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:43.864157 23933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:43.864408 23933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:43.865572 23933 hybrid_clock.cc:648] HybridClock initialized: now 1781996623865561 us; error 40 us; skew 500 ppm
Jun 20 23:03:43 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:43.867947 23933 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:03:43.868314 23933 webserver.cc:460] Webserver started at http://127.17.188.66:38169/ using document root <none> and password file <none>
I20260620 23:03:43.868472 23933 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:43.868510 23933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:43.868590 23933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:43.869578 23933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "d90a950afa5e4f19941f98e4650fdd13"
format_stamp: "Formatted at 2026-06-20 23:03:43 on dist-test-slave-zpss"
server_key: "f59cb030dc9c882c42eb336726acec52"
server_key_iv: "bd3ddbacd0c1a26870706f42c8999e06"
server_key_version: "encryptionkey@0"
I20260620 23:03:43.869871 23933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "d90a950afa5e4f19941f98e4650fdd13"
format_stamp: "Formatted at 2026-06-20 23:03:43 on dist-test-slave-zpss"
server_key: "f59cb030dc9c882c42eb336726acec52"
server_key_iv: "bd3ddbacd0c1a26870706f42c8999e06"
server_key_version: "encryptionkey@0"
I20260620 23:03:43.871140 23933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:03:43.871807 23948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:43.871991 23933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:43.872077 23933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "d90a950afa5e4f19941f98e4650fdd13"
format_stamp: "Formatted at 2026-06-20 23:03:43 on dist-test-slave-zpss"
server_key: "f59cb030dc9c882c42eb336726acec52"
server_key_iv: "bd3ddbacd0c1a26870706f42c8999e06"
server_key_version: "encryptionkey@0"
I20260620 23:03:43.872167 23933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:43.891041 23933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:43.893236 23933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:43.893406 23933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:43.893630 23933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:43.893957 23933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:43.893991 23933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:43.894013 23933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:43.894062 23933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:43.899818 23933 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:36769
I20260620 23:03:43.899887 24061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:36769 every 8 connection(s)
I20260620 23:03:43.900210 23933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:43.900413 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 23933
I20260620 23:03:43.900525 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:43.900753 18161 external_mini_cluster.cc:1428] Setting key dfb69a1af6b6a20668c1194d0c86c678
Jun 20 23:03:43 dist-test-slave-zpss krb5kdc[23692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
I20260620 23:03:43.910135 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:43.901128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:48749 (local address 127.17.188.126:34387)
0620 23:03:43.901190 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:43.901193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:43.901397 (+ 204us) server_negotiation.cc:407] Connection header received
0620 23:03:43.902423 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:43.902424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:43.902437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:43.902500 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:43.903126 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.903629 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.904621 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.904832 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.904919 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:43.907297 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:43.907298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:43.907299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:43.907314 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:43.908695 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:43.909083 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:43.909085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:43.909085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:43.909145 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:43.909421 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:43.909422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:43.909423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:43.909586 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:43.909678 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:43.909990 (+ 312us) server_negotiation.cc:299] Negotiation successful
0620 23:03:43.910063 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:03:43.910383 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:43.901079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34387 (local address 127.17.188.66:48749)
0620 23:03:43.901259 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:43.901304 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:43.902300 (+ 996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:43.902549 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:43.902552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:43.902848 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:43.903031 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:43.903035 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.903732 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.903733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:43.904531 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:43.904533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:43.904928 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:43.904930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:43.905053 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:43.905611 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:43.905619 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:43.907190 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:43.908779 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:43.908786 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:43.908788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:43.909006 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:43.909213 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:43.909214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:43.909215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:43.909366 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:43.909667 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:43.909673 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:43.909816 (+ 143us) client_negotiation.cc:769] Sending connection context
0620 23:03:43.909940 (+ 124us) client_negotiation.cc:240] Negotiation successful
0620 23:03:43.910105 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260620 23:03:43.910862 24062 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34387
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
I20260620 23:03:43.910975 24062 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:43.911166 24062 heartbeater.cc:511] Master 127.17.188.126:34387 requested a full tablet report, sending...
I20260620 23:03:43.911532 23737 ts_manager.cc:194] Registered new tserver with Master: d90a950afa5e4f19941f98e4650fdd13 (127.17.188.66:36769)
I20260620 23:03:43.911912 23737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:48749
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:03:43.925459 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34387
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:44.003408 24067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:44.003594 24067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:44.003618 24067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:44.004653 24067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:44.004689 24067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:44.004712 24067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:44.006012 24067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:44.006168 24067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:44.006320 24067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:44.010954 24075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:44.010967 24072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:44.011159 24067 server_base.cc:1034] running on GCE node
W20260620 23:03:44.010964 24073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:44.011518 24067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:44.011721 24067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:44.012867 24067 hybrid_clock.cc:648] HybridClock initialized: now 1781996624012850 us; error 29 us; skew 500 ppm
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:44.015974 24067 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:03:44.016649 24067 webserver.cc:460] Webserver started at http://127.17.188.67:43119/ using document root <none> and password file <none>
I20260620 23:03:44.016927 24067 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:44.017004 24067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:44.017153 24067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:44.018846 24067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "bf7247103387427787ba402f7b6d1c39"
format_stamp: "Formatted at 2026-06-20 23:03:44 on dist-test-slave-zpss"
server_key: "c3668d844a4d6a0072a7fb6fe10d2a44"
server_key_iv: "7672cb0b550693561b727f4cbf3f5325"
server_key_version: "encryptionkey@0"
I20260620 23:03:44.019418 24067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "bf7247103387427787ba402f7b6d1c39"
format_stamp: "Formatted at 2026-06-20 23:03:44 on dist-test-slave-zpss"
server_key: "c3668d844a4d6a0072a7fb6fe10d2a44"
server_key_iv: "7672cb0b550693561b727f4cbf3f5325"
server_key_version: "encryptionkey@0"
I20260620 23:03:44.021346 24067 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:44.022344 24082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:44.022545 24067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:44.022643 24067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "bf7247103387427787ba402f7b6d1c39"
format_stamp: "Formatted at 2026-06-20 23:03:44 on dist-test-slave-zpss"
server_key: "c3668d844a4d6a0072a7fb6fe10d2a44"
server_key_iv: "7672cb0b550693561b727f4cbf3f5325"
server_key_version: "encryptionkey@0"
I20260620 23:03:44.022727 24067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:44.041038 24067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:44.043558 24067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:44.043715 24067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:44.043927 24067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:44.044238 24067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:44.044276 24067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:44.044317 24067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:44.044346 24067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:44.051553 24067 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:36963
I20260620 23:03:44.051609 24195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:36963 every 8 connection(s)
I20260620 23:03:44.051965 24067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:03:44.061575 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 24067
I20260620 23:03:44.061676 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:44.061872 18161 external_mini_cluster.cc:1428] Setting key e94ca7ae6067402a588dd145cb27006e
I20260620 23:03:44.063400 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:44.052720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:45925 (local address 127.17.188.126:34387)
0620 23:03:44.052769 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:44.052771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:44.052945 (+ 174us) server_negotiation.cc:407] Connection header received
0620 23:03:44.054285 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:44.054287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:44.054299 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:44.054353 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:44.055292 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.056056 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.057575 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.057842 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.057903 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:44.060448 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:44.060450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:44.060450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:44.060458 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:44.061829 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:44.062362 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:44.062363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:44.062363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:44.062432 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:44.062705 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:44.062706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:44.062707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:44.062848 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:44.062910 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:44.063189 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 23:03:44.063313 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 23:03:44.063541 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:44.052687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34387 (local address 127.17.188.67:45925)
0620 23:03:44.052828 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:44.052870 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:44.054097 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:44.054485 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:44.054491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:44.054881 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:44.055190 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:44.055197 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.056150 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.056151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:44.057496 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:44.057499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.057980 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.057981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:44.058138 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:44.058696 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:44.058706 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:44.060356 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:44.061911 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:44.061916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:44.061918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:44.062265 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:44.062501 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:44.062502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:44.062502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:44.062622 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:44.062901 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:44.062907 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:44.063048 (+ 141us) client_negotiation.cc:769] Sending connection context
0620 23:03:44.063156 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 23:03:44.063315 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260620 23:03:44.063941 24196 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34387
I20260620 23:03:44.064041 24196 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:44.064214 24196 heartbeater.cc:511] Master 127.17.188.126:34387 requested a full tablet report, sending...
I20260620 23:03:44.064586 23737 ts_manager.cc:194] Registered new tserver with Master: bf7247103387427787ba402f7b6d1c39 (127.17.188.67:36963)
I20260620 23:03:44.064972 23737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:45925
I20260620 23:03:44.073711 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/test-admin.keytab.
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:44.089299 18161 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 23:03:44.090817 24203 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:03:44.090179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.17.188.65:33411)
0620 23:03:44.090336 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:44.090341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:44.090359 (+ 18us) server_negotiation.cc:407] Connection header received
0620 23:03:44.090383 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:44.090385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:44.090399 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:44.090464 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:44.090760 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55144: BlockingRecv error: recv got EOF from 127.0.0.1:55144 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
W20260620 23:03:44.090984 18161 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5cc)
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:44.091897 18161 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:44.092737 18161 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:44.101016 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:44.093415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55160 (local address 127.17.188.65:33411)
0620 23:03:44.093487 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:44.093489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:44.093514 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:03:44.093534 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:44.093535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:44.093545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:44.093604 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:44.094175 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.095267 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.096067 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.096455 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.096538 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:44.098499 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:44.098507 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:44.098507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:44.098514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:44.099824 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:44.100196 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:44.100199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:44.100200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:44.100261 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:44.100461 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:44.100462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:44.100462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:44.100595 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:44.100730 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:44.100822 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 23:03:44.100896 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260620 23:03:44.102559 24203 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:03:44.102145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.17.188.65:33411)
0620 23:03:44.102181 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:44.102183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:44.102236 (+ 53us) server_negotiation.cc:407] Connection header received
0620 23:03:44.102271 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:44.102272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:44.102279 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:44.102323 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:44.102510 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55172: BlockingRecv error: recv got EOF from 127.0.0.1:55172 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260620 23:03:44.102692 18161 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5cc' of type FILE
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:44.103627 18161 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:44.104429 18161 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:44.111202 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:44.104867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.17.188.65:33411)
0620 23:03:44.104899 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:44.104900 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:44.104983 (+ 83us) server_negotiation.cc:407] Connection header received
0620 23:03:44.105050 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:44.105051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:44.105059 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:44.105105 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:44.105554 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.106023 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.106732 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.106921 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.106992 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:44.108830 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:44.108831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:44.108832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:44.108838 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:44.110067 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:44.110455 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:44.110456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:44.110457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:44.110510 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:44.110704 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:44.110705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:44.110705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:44.110797 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:44.110926 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:44.111027 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 23:03:44.111093 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:03:44.112677 24203 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:03:44.112096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.17.188.65:33411)
0620 23:03:44.112127 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:44.112129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:44.112221 (+ 92us) server_negotiation.cc:407] Connection header received
0620 23:03:44.112304 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:44.112305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:44.112313 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:44.112389 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:44.112631 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55190: BlockingRecv error: recv got EOF from 127.0.0.1:55190 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:03:44.112776 18161 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996599092328-18161-0/krb5kdc/krb5cc)
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:44.113716 18161 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:44.114614 18161 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[23692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:03:44.120906 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:44.115126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55196 (local address 127.17.188.65:33411)
0620 23:03:44.115182 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:44.115184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:44.115256 (+ 72us) server_negotiation.cc:407] Connection header received
0620 23:03:44.115294 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:44.115294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:44.115301 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:44.115345 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:44.115671 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.116087 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.116723 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:44.116925 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:44.116985 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:44.118880 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:44.118881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:44.118881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:44.118886 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:44.120040 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:44.120352 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:44.120352 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:44.120353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:44.120397 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:44.120555 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:44.120555 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:44.120556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:44.120632 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:44.120698 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:44.120778 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:03:44.120836 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 23:03:44.121634 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 23799
I20260620 23:03:44.126603 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 23933
I20260620 23:03:44.131271 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 24067
I20260620 23:03:44.135385 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 23705
2026-06-20T23:03:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2282 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[24216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[24216](info): set up 2 sockets
Jun 20 23:03:44 dist-test-slave-zpss krb5kdc[24216](info): commencing operation
krb5kdc: starting...
W20260620 23:03:45.161567 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:03:45 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.126.keytab.
Entry for principal oryx/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.126.keytab.
I20260620 23:03:45.272208 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42405
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.126.keytab
--principal=oryx/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:45.350780 24229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:45.350958 24229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:45.350979 24229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:45.352077 24229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:45.352108 24229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:45.352140 24229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:45.352152 24229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:45.352170 24229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:45.353606 24229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.126.keytab
--principal=oryx/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:45.353793 24229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:45.354059 24229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:45.358666 24234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:45.358636 24235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:45.358654 24237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:45.358888 24229 server_base.cc:1034] running on GCE node
I20260620 23:03:45.359115 24229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:45.359434 24229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:45.360597 24229 hybrid_clock.cc:648] HybridClock initialized: now 1781996625360584 us; error 46 us; skew 500 ppm
Jun 20 23:03:45 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:45.362946 24229 init.cc:377] Logged in from keytab as oryx/127.17.188.126@KRBTEST.COM (short username oryx)
I20260620 23:03:45.363348 24229 webserver.cc:460] Webserver started at http://127.17.188.126:33443/ using document root <none> and password file <none>
I20260620 23:03:45.363574 24229 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:45.363651 24229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:45.363816 24229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:45.364810 24229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "033c16c578134bf284f736cec5956df6"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "7d06eff91a74860ef56884c73522e2d1"
server_key_iv: "d662494cb64104aa485ebf8ad0b45852"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.365151 24229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "033c16c578134bf284f736cec5956df6"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "7d06eff91a74860ef56884c73522e2d1"
server_key_iv: "d662494cb64104aa485ebf8ad0b45852"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.366453 24229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:45.367131 24244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.367290 24229 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:45.367385 24229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "033c16c578134bf284f736cec5956df6"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "7d06eff91a74860ef56884c73522e2d1"
server_key_iv: "d662494cb64104aa485ebf8ad0b45852"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.367467 24229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:45.404464 24229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:45.406286 24229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:45.406438 24229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:45.410154 24229 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:42405
I20260620 23:03:45.410187 24296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:42405 every 8 connection(s)
I20260620 23:03:45.410501 24229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:45.411432 24297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:45.413565 24297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6: Bootstrap starting.
I20260620 23:03:45.414165 24297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:45.414474 24297 log.cc:824] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:45.415124 24297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6: No bootstrap required, opened a new log
I20260620 23:03:45.416584 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 24229
I20260620 23:03:45.416702 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:03:45.416714 24297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } }
I20260620 23:03:45.416899 18161 external_mini_cluster.cc:1428] Setting key 572cc5d3305eac24df42aeed1f08c8fb
I20260620 23:03:45.416920 24297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:45.416963 24297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 033c16c578134bf284f736cec5956df6, State: Initialized, Role: FOLLOWER
I20260620 23:03:45.417100 24297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } }
I20260620 23:03:45.417197 24297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:45.417238 24297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:45.417279 24297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:45.418361 24297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } }
I20260620 23:03:45.418493 24297 leader_election.cc:302] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 033c16c578134bf284f736cec5956df6; no voters:
I20260620 23:03:45.418655 24297 leader_election.cc:288] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:45.418718 24302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:45.418895 24297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:45.418895 24302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 1 LEADER]: Becoming Leader. State: Replica: 033c16c578134bf284f736cec5956df6, State: Running, Role: LEADER
I20260620 23:03:45.419023 24302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } }
I20260620 23:03:45.419399 24303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "033c16c578134bf284f736cec5956df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } }
I20260620 23:03:45.419507 24303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:45.419423 24304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 033c16c578134bf284f736cec5956df6. Latest consensus state: current_term: 1 leader_uuid: "033c16c578134bf284f736cec5956df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } }
I20260620 23:03:45.419610 24304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:45.419709 24311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:45.420217 24311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:45.422327 24311 catalog_manager.cc:1347] Generated new cluster ID: e7790438fd30452f81920bc2452a953d
I20260620 23:03:45.422376 24311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:03:45 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:03:45.428369 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.417430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.17.188.126:42405)
0620 23:03:45.417589 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:45.417598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:45.417633 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:03:45.417747 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:45.417752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:45.417790 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:45.417901 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:45.418777 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.420262 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.421367 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.421926 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.422064 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.424749 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:45.424753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:45.424753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:45.424765 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:45.426692 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.427067 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.427070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.427070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.427228 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.427427 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.427428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.427429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.427800 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:45.427952 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:45.428029 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 23:03:45.428182 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260620 23:03:45.440389 24311 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:45.440874 24311 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:45.446473 24311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6: Generated new TSK 0
I20260620 23:03:45.446641 24311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab.
Entry for principal oryx/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab.
I20260620 23:03:45.474143 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42405
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab
--principal=oryx/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:45.552121 24323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:45.552305 24323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:45.552327 24323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:45.554031 24323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:45.554073 24323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:45.554100 24323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:45.555492 24323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab
--principal=oryx/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:45.555721 24323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:45.555953 24323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:45.560638 24331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:45.560667 24329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:45.560688 24328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:45.560947 24323 server_base.cc:1034] running on GCE node
I20260620 23:03:45.561133 24323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:45.561406 24323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:45.562568 24323 hybrid_clock.cc:648] HybridClock initialized: now 1781996625562551 us; error 30 us; skew 500 ppm
Jun 20 23:03:45 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:45.564797 24323 init.cc:377] Logged in from keytab as oryx/127.17.188.65@KRBTEST.COM (short username oryx)
I20260620 23:03:45.565182 24323 webserver.cc:460] Webserver started at http://127.17.188.65:33901/ using document root <none> and password file <none>
I20260620 23:03:45.565471 24323 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:45.565541 24323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:45.565677 24323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:45.566612 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "c4aa0a99765345199223664ed6baaaae"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "5731119056da26b464cf35a8b3cc7c6b"
server_key_iv: "1fc718b346d3739c9ff4286fbc3dbd48"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.566946 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "c4aa0a99765345199223664ed6baaaae"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "5731119056da26b464cf35a8b3cc7c6b"
server_key_iv: "1fc718b346d3739c9ff4286fbc3dbd48"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.568261 24323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:45.568893 24338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.569042 24323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:45.569132 24323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "c4aa0a99765345199223664ed6baaaae"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "5731119056da26b464cf35a8b3cc7c6b"
server_key_iv: "1fc718b346d3739c9ff4286fbc3dbd48"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.569212 24323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:45.588164 24323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:45.590025 24323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:45.590183 24323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:45.590390 24323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:45.590680 24323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:45.590735 24323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.590775 24323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:45.590802 24323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.596050 24323 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:34825
I20260620 23:03:45.596127 24451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:34825 every 8 connection(s)
I20260620 23:03:45.596406 24323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:45.598208 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 24323
I20260620 23:03:45.598318 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:45.598500 18161 external_mini_cluster.cc:1428] Setting key 7d1b3bba7cf00c9e4ee51f8299e65641
Jun 20 23:03:45 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:03:45.605885 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.597047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:54227 (local address 127.17.188.126:42405)
0620 23:03:45.597094 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:45.597096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:45.597328 (+ 232us) server_negotiation.cc:407] Connection header received
0620 23:03:45.597809 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:45.597811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:45.597820 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:45.597864 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:45.598820 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.599439 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.600416 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.600635 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.600728 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.602930 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:45.602931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:45.602932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:45.602940 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:45.604542 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.604917 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.604919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.604919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.604979 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.605192 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.605192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.605193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.605361 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:45.605440 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:45.605666 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 23:03:45.605749 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:03:45.605906 24454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.597038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:54227)
0620 23:03:45.597195 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:45.597226 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:45.597718 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:45.597922 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:45.597925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:45.598538 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:45.598758 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:45.598762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.599531 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.599532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:45.600324 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:45.600326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.600736 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.600737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:45.600870 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.601400 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:45.601409 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:45.602842 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:45.604616 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:45.604622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:45.604624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:45.604847 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:45.605042 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:45.605043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:45.605043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:45.605128 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:45.605433 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:45.605437 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:45.605541 (+ 104us) client_negotiation.cc:769] Sending connection context
0620 23:03:45.605620 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:03:45.605727 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260620 23:03:45.606400 24452 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42405
I20260620 23:03:45.606493 24452 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:45.606675 24452 heartbeater.cc:511] Master 127.17.188.126:42405 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.17.188.66@KRBTEST.COM; defaulting to no policy
I20260620 23:03:45.607187 24261 ts_manager.cc:194] Registered new tserver with Master: c4aa0a99765345199223664ed6baaaae (127.17.188.65:34825)
I20260620 23:03:45.607762 24261 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:54227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab.
Entry for principal oryx/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab.
I20260620 23:03:45.623791 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42405
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab
--principal=oryx/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:45.702294 24457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:45.702486 24457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:45.702507 24457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:45.703634 24457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:45.703673 24457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:45.703720 24457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:45.705082 24457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab
--principal=oryx/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:45.705289 24457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:45.705523 24457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:45.710206 24465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:45.710273 24463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:45.710341 24457 server_base.cc:1034] running on GCE node
W20260620 23:03:45.710206 24462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:45.710600 24457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:45.710822 24457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:45.712101 24457 hybrid_clock.cc:648] HybridClock initialized: now 1781996625712093 us; error 40 us; skew 500 ppm
Jun 20 23:03:45 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:45.714457 24457 init.cc:377] Logged in from keytab as oryx/127.17.188.66@KRBTEST.COM (short username oryx)
I20260620 23:03:45.714856 24457 webserver.cc:460] Webserver started at http://127.17.188.66:36213/ using document root <none> and password file <none>
I20260620 23:03:45.715066 24457 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:45.715107 24457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:45.715229 24457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:45.716158 24457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "54527a31bff649169685d8a02bfe7ac4"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "3a6a28793739feb4225b41ffda2bf892"
server_key_iv: "bbb4aa520610fdc351f28d484e9ce82a"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.716491 24457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "54527a31bff649169685d8a02bfe7ac4"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "3a6a28793739feb4225b41ffda2bf892"
server_key_iv: "bbb4aa520610fdc351f28d484e9ce82a"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.717875 24457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:03:45.718565 24472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.718737 24457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:45.718819 24457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "54527a31bff649169685d8a02bfe7ac4"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "3a6a28793739feb4225b41ffda2bf892"
server_key_iv: "bbb4aa520610fdc351f28d484e9ce82a"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.718887 24457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:45.732981 24457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:45.734664 24457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:45.734819 24457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:45.735014 24457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:45.735294 24457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:45.735345 24457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.735384 24457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:45.735411 24457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.741050 24457 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:45627
I20260620 23:03:45.741087 24585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:45627 every 8 connection(s)
I20260620 23:03:45.741456 24457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:03:45 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:03:45.748045 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 24457
I20260620 23:03:45.748165 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:45.748378 18161 external_mini_cluster.cc:1428] Setting key 104002531d13d49e08716bd5f001d2b8
I20260620 23:03:45.750272 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.742113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:36923 (local address 127.17.188.126:42405)
0620 23:03:45.742175 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:45.742176 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:45.742372 (+ 196us) server_negotiation.cc:407] Connection header received
0620 23:03:45.742842 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:45.742843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:45.742855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:45.742918 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:45.743771 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.744278 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.745239 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.745516 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.745617 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.747459 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:45.747460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:45.747460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:45.747469 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:45.748932 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.749297 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.749298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.749299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.749378 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.749610 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.749612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.749612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.749756 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:45.749820 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:45.750056 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 23:03:45.750145 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:03:45.750291 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.742109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:36923)
0620 23:03:45.742257 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:45.742291 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:45.742727 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:45.742987 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:45.742990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:45.743508 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:45.743670 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:45.743674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.744368 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.744369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:45.745148 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:45.745150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.745625 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.745626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:45.745742 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.746127 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:45.746134 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:45.747365 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:45.749008 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:45.749014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:45.749016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:45.749225 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:45.749449 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:45.749449 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:45.749449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:45.749541 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:45.749815 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:45.749818 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:45.749937 (+ 119us) client_negotiation.cc:769] Sending connection context
0620 23:03:45.750013 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:03:45.750116 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260620 23:03:45.750922 24586 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42405
I20260620 23:03:45.750995 24586 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:45.751179 24586 heartbeater.cc:511] Master 127.17.188.126:42405 requested a full tablet report, sending...
I20260620 23:03:45.751547 24261 ts_manager.cc:194] Registered new tserver with Master: 54527a31bff649169685d8a02bfe7ac4 (127.17.188.66:45627)
I20260620 23:03:45.751884 24261 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:36923
WARNING: no policy specified for oryx/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab.
Entry for principal oryx/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab.
I20260620 23:03:45.771988 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42405
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab
--principal=oryx/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:45.853600 24591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:45.853849 24591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:45.853888 24591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:45.855014 24591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:45.855078 24591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:45.855160 24591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:45.856570 24591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab
--principal=oryx/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:45.856812 24591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:45.857016 24591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:45.861702 24597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:45.861874 24591 server_base.cc:1034] running on GCE node
W20260620 23:03:45.861703 24599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:45.861758 24596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:45.862143 24591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:45.862372 24591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:45.863543 24591 hybrid_clock.cc:648] HybridClock initialized: now 1781996625863525 us; error 29 us; skew 500 ppm
Jun 20 23:03:45 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:45.865963 24591 init.cc:377] Logged in from keytab as oryx/127.17.188.67@KRBTEST.COM (short username oryx)
I20260620 23:03:45.866362 24591 webserver.cc:460] Webserver started at http://127.17.188.67:44017/ using document root <none> and password file <none>
I20260620 23:03:45.866587 24591 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:45.866649 24591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:45.866794 24591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:45.867838 24591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "4d89b4354fb740cc84e8ff02e6f0b383"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "f81a4c1f046bbd24c8862833cd2542c0"
server_key_iv: "246cf51892cf8277fbeeb007df3f67a3"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.868172 24591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "4d89b4354fb740cc84e8ff02e6f0b383"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "f81a4c1f046bbd24c8862833cd2542c0"
server_key_iv: "246cf51892cf8277fbeeb007df3f67a3"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.869560 24591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:03:45.870245 24606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.870451 24591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.870544 24591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "4d89b4354fb740cc84e8ff02e6f0b383"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "f81a4c1f046bbd24c8862833cd2542c0"
server_key_iv: "246cf51892cf8277fbeeb007df3f67a3"
server_key_version: "encryptionkey@0"
I20260620 23:03:45.870613 24591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:45.880116 24591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:45.881899 24591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:45.882045 24591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:45.882241 24591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:45.882500 24591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:45.882550 24591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.882586 24591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:45.882613 24591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:45.888005 24591 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:33855
I20260620 23:03:45.888057 24719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:33855 every 8 connection(s)
I20260620 23:03:45.888345 24591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:03:45 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:03:45.896317 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 24591
I20260620 23:03:45.896418 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:45.896479 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.888977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:53627 (local address 127.17.188.126:42405)
0620 23:03:45.889034 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:45.889036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:45.889237 (+ 201us) server_negotiation.cc:407] Connection header received
0620 23:03:45.889679 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:45.889680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:45.889689 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:45.889743 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:45.890597 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.891058 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.891935 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.892122 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.892170 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.894076 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:45.894077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:45.894077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:45.894083 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:45.895356 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.895670 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.895670 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.895671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.895725 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.895919 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.895919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.895920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.896012 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:45.896058 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:45.896289 (+ 231us) server_negotiation.cc:299] Negotiation successful
0620 23:03:45.896367 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:03:45.896618 18161 external_mini_cluster.cc:1428] Setting key d23066352e41970ee2ac0219e70f68ea
I20260620 23:03:45.896620 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.888959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:53627)
0620 23:03:45.889125 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:45.889155 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:45.889602 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:45.889792 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:45.889795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:45.890355 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:45.890529 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:45.890533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.891129 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.891130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:45.891871 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:45.891873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.892187 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.892188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:45.892279 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.892667 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:45.892674 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:45.893999 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:45.895417 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:45.895421 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:45.895422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:45.895606 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:45.895785 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:45.895786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:45.895786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:45.895869 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:45.896062 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:45.896065 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:45.896162 (+ 97us) client_negotiation.cc:769] Sending connection context
0620 23:03:45.896236 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 23:03:45.896340 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260620 23:03:45.897100 24720 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42405
I20260620 23:03:45.897207 24720 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:45.897418 24720 heartbeater.cc:511] Master 127.17.188.126:42405 requested a full tablet report, sending...
I20260620 23:03:45.897791 24261 ts_manager.cc:194] Registered new tserver with Master: 4d89b4354fb740cc84e8ff02e6f0b383 (127.17.188.67:33855)
I20260620 23:03:45.898101 24261 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:53627
I20260620 23:03:45.908425 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.188.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.125.keytab.
Entry for principal oryx/127.17.188.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.125.keytab.
I20260620 23:03:45.931576 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.125:36513
--webserver_interface=127.17.188.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.125.keytab
--principal=oryx/127.17.188.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42405,127.17.188.125:36513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:46.007993 24725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:46.008179 24725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:46.008200 24725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:46.009225 24725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:46.009256 24725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:46.009270 24725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:46.009282 24725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:46.009294 24725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:46.010689 24725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42405,127.17.188.125:36513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.125.keytab
--principal=oryx/127.17.188.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.125:36513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.188.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
W20260620 23:03:46.010785 24725 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.188.126:42405,127.17.188.125:36513'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 23:03:46.010881 24725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:46.011027 24725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:46.015544 24733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.015596 24731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.015543 24730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:46.015744 24725 server_base.cc:1034] running on GCE node
I20260620 23:03:46.015942 24725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:46.016215 24725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:46.017373 24725 hybrid_clock.cc:648] HybridClock initialized: now 1781996626017364 us; error 29 us; skew 500 ppm
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:46.019969 24725 init.cc:377] Logged in from keytab as oryx/127.17.188.125@KRBTEST.COM (short username oryx)
I20260620 23:03:46.020398 24725 webserver.cc:460] Webserver started at http://127.17.188.125:37991/ using document root <none> and password file <none>
I20260620 23:03:46.020633 24725 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:46.020699 24725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:46.020839 24725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:46.021930 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data/instance:
uuid: "7bddbce09fac4a67a49021ec992db0a7"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "60a25e71aeced18c4943207b404a4fee"
server_key_iv: "5bbcde1077424a69dbcdebd05cf8c4e5"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.022269 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal/instance:
uuid: "7bddbce09fac4a67a49021ec992db0a7"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "60a25e71aeced18c4943207b404a4fee"
server_key_iv: "5bbcde1077424a69dbcdebd05cf8c4e5"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.023571 24725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:03:46.024235 24740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.024394 24725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:46.024477 24725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal
uuid: "7bddbce09fac4a67a49021ec992db0a7"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "60a25e71aeced18c4943207b404a4fee"
server_key_iv: "5bbcde1077424a69dbcdebd05cf8c4e5"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.024550 24725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:46.036789 24725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:46.038491 24725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:46.038640 24725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:46.042356 24725 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.125:36513
I20260620 23:03:46.042387 24792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.125:36513 every 8 connection(s)
I20260620 23:03:46.042675 24725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
I20260620 23:03:46.043529 24793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:46.044875 24793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:03:46.045760 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 24725
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.125@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.188.125@KRBTEST.COM
I20260620 23:03:46.054036 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.045516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33276 (local address 127.17.188.126:42405)
0620 23:03:46.045598 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.045600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.045766 (+ 166us) server_negotiation.cc:407] Connection header received
0620 23:03:46.046311 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.046312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.046327 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.046382 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.046947 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.047443 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.048505 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.048715 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.048799 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.050996 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.050998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.050998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.051006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.052613 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.053050 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.053051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.053051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.053113 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.053389 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.053390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.053390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.053540 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.053619 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.053867 (+ 248us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.053944 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:03:46.054293 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.046243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.17.188.125:36513)
0620 23:03:46.046430 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.046435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.046485 (+ 50us) server_negotiation.cc:407] Connection header received
0620 23:03:46.046542 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.046545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.046568 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.046671 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.047259 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.048051 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.048756 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.049060 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.049124 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.051466 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.051473 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.051473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.051480 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.052925 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.053296 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.053299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.053299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.053400 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.053598 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.053599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.053600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.053751 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.053928 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.054010 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.054087 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260620 23:03:46.054299 24795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.045480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:33276)
0620 23:03:46.045654 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.045691 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.046149 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.046483 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.046488 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.046715 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.046853 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.046857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.047553 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.047554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.048404 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.048406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.048806 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.048807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.048919 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.049408 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.049417 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.050906 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.052700 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.052706 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.052707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.052974 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.053180 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.053181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.053181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.053293 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.053610 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.053615 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.053747 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.053847 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.053984 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":153,"thread_start_us":86,"threads_started":1}
I20260620 23:03:46.054811 24793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.125@KRBTEST.COM for oryx/127.17.188.125@KRBTEST.COM
I20260620 23:03:46.060149 24795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.055069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:36513 (local address 127.0.0.1:35076)
0620 23:03:46.055096 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.055105 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.055139 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.055235 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.055236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.055333 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.055370 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.055371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.055913 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.055914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.056355 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.056356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.056650 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.056651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.056697 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.057084 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.057088 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.058230 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.059450 (+ 1220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.059450 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.059451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.059599 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.059745 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.059746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.059746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.059786 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.059929 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.059930 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.059976 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.060021 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.060082 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13}
I20260620 23:03:46.060245 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.055099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35076 (local address 127.17.188.125:36513)
0620 23:03:46.055145 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.055146 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.055155 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:03:46.055175 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.055175 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.055183 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.055223 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.055421 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.055849 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.056419 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.056593 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.056639 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.058332 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.058333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.058333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.058339 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.059389 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.059656 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.059657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.059657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.059699 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.059823 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.059823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.059823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.059896 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.059936 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.060033 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.060121 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":31}
I20260620 23:03:46.061250 24793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: Bootstrap starting.
I20260620 23:03:46.062007 24793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:46.062264 24793 log.cc:824] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:46.062881 24793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: No bootstrap required, opened a new log
I20260620 23:03:46.064127 24793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } }
I20260620 23:03:46.064251 24793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:46.064324 24793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bddbce09fac4a67a49021ec992db0a7, State: Initialized, Role: FOLLOWER
I20260620 23:03:46.064462 24793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } }
I20260620 23:03:46.064739 24793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:46.064709 24799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } } }
I20260620 23:03:46.064810 24799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:03:46.066366 24810 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:03:46.066412 24810 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:03:46.067704 24725 master_runner.cc:428] Detected that this master 7bddbce09fac4a67a49021ec992db0a7 is joining an existing cluster
I20260620 23:03:46.067747 24725 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.188.125:36513
I20260620 23:03:46.075045 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.068761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33282 (local address 127.17.188.126:42405)
0620 23:03:46.068799 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.068801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.068906 (+ 105us) server_negotiation.cc:407] Connection header received
0620 23:03:46.068997 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.068998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.069008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.069067 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.069457 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.070014 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.070825 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.071079 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.071164 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.072108 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.072109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.072109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.072115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.073810 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.074260 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.074261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.074262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.074322 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.074534 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.074535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.074536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.074644 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.074728 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.074879 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.074947 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:03:46.075062 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.068646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:33282)
0620 23:03:46.068805 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.068844 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.068931 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.069084 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.069084 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.069252 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.069381 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.069383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.070140 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.070141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.070726 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.070728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.071176 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.071177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.071239 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.071682 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.071687 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.072029 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.073942 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.073943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.073943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.074146 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.074386 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.074387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.074387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.074446 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.074709 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.074710 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.074750 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.074854 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.074948 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260620 23:03:46.076412 24817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.069284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:36513 (local address 127.0.0.1:35086)
0620 23:03:46.069481 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.069490 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.069555 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.069735 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.069736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.069919 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.069972 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.069973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.070817 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.070818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.071631 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.071634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.072067 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.072068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.072132 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.072559 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.072564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.072905 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.075006 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.075007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.075007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.075566 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.075774 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.075774 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.075775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.075846 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.076093 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.076093 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.076136 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.076219 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.076302 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260620 23:03:46.076597 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.069215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35086 (local address 127.17.188.125:36513)
0620 23:03:46.069268 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.069270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.069528 (+ 258us) server_negotiation.cc:407] Connection header received
0620 23:03:46.069614 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.069615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.069622 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.069646 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.070049 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.070697 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.071708 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.071986 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.072056 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.072992 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.072993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.072993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.072999 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.074876 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.075648 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.075649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.075649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.075695 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.075909 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.075910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.075910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.076021 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.076056 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.076278 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.076507 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.126@KRBTEST.COM for oryx/127.17.188.125@KRBTEST.COM
I20260620 23:03:46.083729 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.077011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35088 (local address 127.17.188.125:36513)
0620 23:03:46.077040 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.077042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.077501 (+ 459us) server_negotiation.cc:407] Connection header received
0620 23:03:46.077597 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.077598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.077605 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.077645 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.077991 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.078390 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.079116 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.079313 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.079384 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.081210 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.081211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.081211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.081217 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.082590 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.082971 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.082972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.082973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.083026 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.083244 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.083244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.083245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.083343 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.083408 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.083577 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.083642 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:03:46.083741 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.076977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:36513 (local address 127.0.0.1:35088)
0620 23:03:46.077397 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.077430 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.077520 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.077647 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.077650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.077877 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.077912 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.077913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.078470 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.078471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.079035 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.079037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.079385 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.079385 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.079485 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.079935 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.079940 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.081115 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.082687 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.082694 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.082696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.082884 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.083104 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.083105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.083105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.083159 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.083415 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.083422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.083465 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.083555 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.083652 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":55,"threads_started":1}
I20260620 23:03:46.084172 24261 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.188.125:36513: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7"
member_type: NON_VOTER
last_known_addr {
host: "127.17.188.125"
port: 36513
}
attrs {
promote: true
}
}
dest_uuid: "033c16c578134bf284f736cec5956df6"
cas_config_opid_index: -1
I20260620 23:03:46.084467 24261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } }
I20260620 23:03:46.084872 24820 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7bddbce09fac4a67a49021ec992db0a7 (127.17.188.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } } }
I20260620 23:03:46.085119 24767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:46.085739 24820 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.188.125:36513
I20260620 23:03:46.085829 24820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "033c16c578134bf284f736cec5956df6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } } }
I20260620 23:03:46.085891 24820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:46.085945 24725 master.cc:537] Master@127.17.188.125:36513 shutting down...
I20260620 23:03:46.085968 24767 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Refusing update from remote peer 033c16c578134bf284f736cec5956df6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 23:03:46.086990 24819 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:03:46.087692 24248 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260620 23:03:46.088120 24248 consensus_peers.cc:487] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 -> Peer 7bddbce09fac4a67a49021ec992db0a7 (127.17.188.125:36513): Couldn't send request to peer 7bddbce09fac4a67a49021ec992db0a7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 23:03:46.088874 24725 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:46.088950 24725 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:46.088994 24725 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: stopping tablet replica
I20260620 23:03:46.101349 24725 master.cc:552] Master@127.17.188.125:36513 shutdown complete.
I20260620 23:03:46.101399 24725 master_runner.cc:305] Clearing existing system tablet
I20260620 23:03:46.101523 24725 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:46.101567 24725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:46.102240 24725 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:03:46.102727 24824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.102890 24725 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:46.102950 24725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.102975 24725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal
uuid: "7bddbce09fac4a67a49021ec992db0a7"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "60a25e71aeced18c4943207b404a4fee"
server_key_iv: "5bbcde1077424a69dbcdebd05cf8c4e5"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.103699 24725 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:46.105140 24725 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 23:03:46.105225 24725 log.cc:1196] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 23:03:46.105471 24725 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: Deleting consensus metadata
I20260620 23:03:46.105626 24725 master_runner.cc:315] Copying system tablet from 127.17.188.126:42405
I20260620 23:03:46.106189 24725 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: tablet copy: Beginning tablet copy session from remote peer at address 127.17.188.126:42405
I20260620 23:03:46.113137 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.106539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.17.188.126:42405)
0620 23:03:46.106586 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.106587 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.106631 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:03:46.106717 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.106718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.106729 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.106779 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.107428 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.107895 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.108925 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.109089 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.109135 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.110116 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.110117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.110118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.110124 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.111702 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.112196 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.112197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.112197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.112263 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.112475 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.112476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.112476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.112605 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.112654 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.112919 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.113016 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 23:03:46.113526 24830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.106452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:33290)
0620 23:03:46.106565 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.106575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.106640 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.106802 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.106803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.106998 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.107350 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.107353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.107971 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.107972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.108848 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.108850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.109153 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.109154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.109218 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.109635 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.109640 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.110033 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.111838 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.111840 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.111840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.112107 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.112337 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.112338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.112338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.112400 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.112708 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.112709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.112795 (+ 86us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.113279 (+ 484us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.113417 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":76,"threads_started":1}
I20260620 23:03:46.114071 24281 tablet_copy_service.cc:140] P 033c16c578134bf284f736cec5956df6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7bddbce09fac4a67a49021ec992db0a7 ({username='oryx', principal='oryx/127.17.188.125@KRBTEST.COM'} at 127.0.0.1:33290)
I20260620 23:03:46.114178 24281 tablet_copy_service.cc:161] P 033c16c578134bf284f736cec5956df6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7bddbce09fac4a67a49021ec992db0a7 at {username='oryx', principal='oryx/127.17.188.125@KRBTEST.COM'} at 127.0.0.1:33290: session id = 7bddbce09fac4a67a49021ec992db0a7-00000000000000000000000000000000
I20260620 23:03:46.114758 24281 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 23:03:46.115265 24725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 23:03:46.116082 24725 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: tablet copy: Starting download of 0 data blocks...
I20260620 23:03:46.116252 24725 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: tablet copy: Starting download of 1 WAL segments...
I20260620 23:03:46.117116 24725 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 23:03:46.117864 24281 tablet_copy_service.cc:342] P 033c16c578134bf284f736cec5956df6: Request end of tablet copy session 7bddbce09fac4a67a49021ec992db0a7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.188.125@KRBTEST.COM'} at 127.0.0.1:33290
I20260620 23:03:46.117924 24281 tablet_copy_service.cc:433] P 033c16c578134bf284f736cec5956df6: ending tablet copy session 7bddbce09fac4a67a49021ec992db0a7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7bddbce09fac4a67a49021ec992db0a7
I20260620 23:03:46.120762 24725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:46.121857 24831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.121927 24832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.122079 24834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:46.122321 24725 server_base.cc:1034] running on GCE node
I20260620 23:03:46.122421 24725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:46.122546 24725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:46.123657 24725 hybrid_clock.cc:648] HybridClock initialized: now 1781996626123647 us; error 26 us; skew 500 ppm
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:46.124821 24725 init.cc:377] Logged in from keytab as oryx/127.17.188.125@KRBTEST.COM (short username oryx)
I20260620 23:03:46.125092 24725 webserver.cc:460] Webserver started at http://127.17.188.125:39305/ using document root <none> and password file <none>
I20260620 23:03:46.125228 24725 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:46.125265 24725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:46.125867 24725 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.126362 24841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.126487 24725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:46.126531 24725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal
uuid: "7bddbce09fac4a67a49021ec992db0a7"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "60a25e71aeced18c4943207b404a4fee"
server_key_iv: "5bbcde1077424a69dbcdebd05cf8c4e5"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.126581 24725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:46.146982 24725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:46.148413 24725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:46.151381 24725 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.125:36513
I20260620 23:03:46.151510 24893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.125:36513 every 8 connection(s)
I20260620 23:03:46.151698 24725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
I20260620 23:03:46.152280 24894 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:03:46.152640 24894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: Bootstrap starting.
I20260620 23:03:46.154736 24894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:03:46.154958 24894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: Bootstrap complete.
I20260620 23:03:46.155151 24894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } }
I20260620 23:03:46.155212 24894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7bddbce09fac4a67a49021ec992db0a7, State: Initialized, Role: LEARNER
I20260620 23:03:46.155238 24894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } }
I20260620 23:03:46.155409 24894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:46.155406 24897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } } }
I20260620 23:03:46.155746 24897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: This master's current role is: LEARNER
I20260620 23:03:46.156917 24909 catalog_manager.cc:1259] Loaded cluster ID: e7790438fd30452f81920bc2452a953d
I20260620 23:03:46.157286 24909 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: loading cluster ID for follower catalog manager: success
I20260620 23:03:46.158582 24909 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: acquiring CA information for follower catalog manager: success
I20260620 23:03:46.158880 24909 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:03:46.164367 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.155879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35102 (local address 127.17.188.125:36513)
0620 23:03:46.156757 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.156759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.156767 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:46.156788 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.156789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.156799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.157828 (+ 1029us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.157859 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.158754 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.159751 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.160015 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.160070 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.161478 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.161479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.161479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.161485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.162918 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.163348 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.163349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.163349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.163397 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.163598 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.163599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.163599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.163689 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.163766 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.164143 (+ 377us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.164246 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for oryx/127.17.188.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.124.keytab.
Entry for principal oryx/127.17.188.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.124.keytab.
I20260620 23:03:46.188479 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.124:38545
--webserver_interface=127.17.188.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.124.keytab
--principal=oryx/127.17.188.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:46.267465 24912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:46.267665 24912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:46.267700 24912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:46.268785 24912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:46.268850 24912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:46.268875 24912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:46.268904 24912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:46.268927 24912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:46.270498 24912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.124.keytab
--principal=oryx/127.17.188.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.124:38545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.188.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:46.270744 24912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:46.270948 24912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:03:46.275646 24912 server_base.cc:1034] running on GCE node
W20260620 23:03:46.275601 24918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.275631 24917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.275592 24920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:46.276093 24912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:46.276392 24912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:46.277560 24912 hybrid_clock.cc:648] HybridClock initialized: now 1781996626277557 us; error 34 us; skew 500 ppm
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:46.279920 24912 init.cc:377] Logged in from keytab as oryx/127.17.188.124@KRBTEST.COM (short username oryx)
I20260620 23:03:46.280355 24912 webserver.cc:460] Webserver started at http://127.17.188.124:34495/ using document root <none> and password file <none>
I20260620 23:03:46.280558 24912 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:46.280622 24912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:46.280773 24912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:46.281826 24912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data/instance:
uuid: "1b0cd1229f0b4b94811801f7b83342b1"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "3fb60f0180c98fd4bf3e823269a7caaa"
server_key_iv: "df9ce44b2a4b4a17e4636b1d21f7d0a2"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.282160 24912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal/instance:
uuid: "1b0cd1229f0b4b94811801f7b83342b1"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "3fb60f0180c98fd4bf3e823269a7caaa"
server_key_iv: "df9ce44b2a4b4a17e4636b1d21f7d0a2"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.283464 24912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:03:46.284132 24927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.284312 24912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:46.284403 24912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal
uuid: "1b0cd1229f0b4b94811801f7b83342b1"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "3fb60f0180c98fd4bf3e823269a7caaa"
server_key_iv: "df9ce44b2a4b4a17e4636b1d21f7d0a2"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.284479 24912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:46.300695 24912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:46.302894 24912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:46.303038 24912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:46.306882 24912 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.124:38545
I20260620 23:03:46.306910 24979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.124:38545 every 8 connection(s)
I20260620 23:03:46.307260 24912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
I20260620 23:03:46.308117 24980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:46.309725 24980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:03:46.312963 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 24912
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.124@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:03:46.318547 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.310554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33306 (local address 127.17.188.126:42405)
0620 23:03:46.310619 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.310621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.310685 (+ 64us) server_negotiation.cc:407] Connection header received
0620 23:03:46.311102 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.311103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.311112 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.311163 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.311676 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.312143 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.313023 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.313244 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.313319 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.315586 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.315587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.315587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.315595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.317039 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.317529 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.317531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.317531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.317586 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.317831 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.317832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.317833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.317997 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.318077 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.318296 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.318400 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:03:46.318620 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.310401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:33306)
0620 23:03:46.310579 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.310623 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.311024 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.311222 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.311225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.311441 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.311611 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.311614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.312220 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.312221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.312926 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.312928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.313381 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.313383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.313543 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.314066 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.314074 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.315518 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.317133 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.317138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.317140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.317462 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.317653 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.317653 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.317653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.317754 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.318068 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.318071 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.318168 (+ 97us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.318246 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.318407 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260620 23:03:46.319213 24980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.188.124@KRBTEST.COM
I20260620 23:03:46.323065 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.313481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39446 (local address 127.17.188.124:38545)
0620 23:03:46.313654 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.313658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.313678 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:03:46.313704 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.313706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.313720 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.313845 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.314407 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.315424 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.316537 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.316827 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.316877 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.320015 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.320021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.320022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.320029 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.321827 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.322209 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.322211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.322211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.322261 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.322518 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.322519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.322519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.322643 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.322781 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.322877 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.322949 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.124@KRBTEST.COM for oryx/127.17.188.125@KRBTEST.COM
I20260620 23:03:46.325798 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.319510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:36513 (local address 127.0.0.1:35112)
0620 23:03:46.319561 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.319572 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.319610 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.319731 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.319732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.319863 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.319911 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.319913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.320868 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.320869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.321578 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.321580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.322076 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.322077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.322155 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.322619 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.322623 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.323775 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.325006 (+ 1231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.325006 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.325007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.325149 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.325352 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.325352 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.325353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.325413 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.325605 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.325606 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.325635 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.325682 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.325747 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260620 23:03:46.325824 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.319565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35112 (local address 127.17.188.125:36513)
0620 23:03:46.319608 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.319610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.319619 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:03:46.319666 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.319667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.319682 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.319724 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.320009 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.320789 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.321659 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.322006 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.322055 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.323847 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.323848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.323848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.323854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.324942 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.325206 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.325207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.325207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.325252 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.325456 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.325457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.325457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.325554 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.325603 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.325697 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.325758 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:03:46.326261 24980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38545 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.124@KRBTEST.COM for oryx/127.17.188.124@KRBTEST.COM
I20260620 23:03:46.331748 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.326453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38545 (local address 127.0.0.1:39462)
0620 23:03:46.326496 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.326506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.326546 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.326647 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.326648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.326770 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.326790 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.326791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.327360 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.327361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.327794 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.327795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.328163 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.328164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.328209 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.328549 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.328553 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.329575 (+ 1022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.331066 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.331066 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.331067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.331213 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.331354 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.331355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.331355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.331398 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.331549 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.331549 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.331577 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.331622 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.331680 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 23:03:46.331795 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.326528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39462 (local address 127.17.188.124:38545)
0620 23:03:46.326556 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.326558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.326566 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:46.326585 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.326586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.326594 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.326645 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.326863 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.327304 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.327880 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.328081 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.328163 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.329663 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.329664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.329664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.329669 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.331001 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.331268 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.331268 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.331269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.331306 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.331432 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.331432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.331432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.331504 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.331544 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.331636 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.331717 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":14}
I20260620 23:03:46.332909 24980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: Bootstrap starting.
I20260620 23:03:46.333698 24980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:46.333974 24980 log.cc:824] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:46.334604 24980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: No bootstrap required, opened a new log
I20260620 23:03:46.335903 24980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38545 } }
I20260620 23:03:46.336033 24980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:46.336077 24980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0cd1229f0b4b94811801f7b83342b1, State: Initialized, Role: FOLLOWER
I20260620 23:03:46.336205 24980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38545 } }
I20260620 23:03:46.336452 24985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38545 } } }
I20260620 23:03:46.336503 24980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:46.336548 24985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:03:46.338021 24996 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:03:46.338106 24996 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:03:46.339985 24912 master_runner.cc:428] Detected that this master 1b0cd1229f0b4b94811801f7b83342b1 is joining an existing cluster
I20260620 23:03:46.340031 24912 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.188.124:38545
I20260620 23:03:46.347663 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.341553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:36513 (local address 127.0.0.1:35116)
0620 23:03:46.341659 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.341669 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.341806 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.341982 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.341983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.342193 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.342257 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.342259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.343161 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.343161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.344039 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.344041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.344517 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.344518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.344581 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.344956 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.344960 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.345213 (+ 253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.346799 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.346799 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.346800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.346945 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.347144 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.347145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.347145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.347218 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.347470 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.347470 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.347498 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.347540 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.347602 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":42,"thread_start_us":71,"threads_started":1}
I20260620 23:03:46.347718 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.341247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.17.188.125:36513)
0620 23:03:46.341281 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.341282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.341712 (+ 430us) server_negotiation.cc:407] Connection header received
0620 23:03:46.341885 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.341886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.341895 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.341925 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.342343 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.343079 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.344128 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.344419 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.344454 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.345301 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.345302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.345302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.345336 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.346725 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.347015 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.347016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.347017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.347074 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.347281 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.347282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.347282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.347407 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.347480 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.347563 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.347634 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:03:46.364763 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.341044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.17.188.126:42405)
0620 23:03:46.341085 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.341086 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.341197 (+ 111us) server_negotiation.cc:407] Connection header received
0620 23:03:46.341244 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.341245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.341255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.341367 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.341752 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.342455 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.343543 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.343840 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.343879 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.361705 (+ 17826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.361706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.361707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.361713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.363424 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.363857 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.363858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.363859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.363906 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.364182 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.364183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.364183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.364335 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.364419 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.364598 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.364674 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:46.364903 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.341752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.17.188.124:38545)
0620 23:03:46.341786 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.341788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.341904 (+ 116us) server_negotiation.cc:407] Connection header received
0620 23:03:46.341981 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.341981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.341988 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.342011 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.342393 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.343064 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.344127 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.344416 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.344453 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.362155 (+ 17702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.362156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.362156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.362163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.363720 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.364074 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.364075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.364075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.364125 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.364317 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.364318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.364318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.364452 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.364551 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.364686 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.364788 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":19}
I20260620 23:03:46.365104 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.340984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:33314)
0620 23:03:46.341117 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.341138 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.341192 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.341410 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.341410 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.341590 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.341661 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.341662 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.342569 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.342570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.343442 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.343445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.343959 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.343960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.344017 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.344447 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.344452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.361604 (+ 17152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.363520 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.363521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.363521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.363794 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.363997 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.363998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.363998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.364060 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.364421 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.364422 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.364465 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.364869 (+ 404us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.364961 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":84,"threads_started":1}
I20260620 23:03:46.365341 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.341742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38545 (local address 127.0.0.1:39464)
0620 23:03:46.341856 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.341864 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.341922 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.342057 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.342057 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.342271 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.342327 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.342328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.343146 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.343147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.344039 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.344041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.344517 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.344518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.344582 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.361730 (+ 17148us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.361735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.362080 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.363804 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.363805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.363805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.364025 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.364203 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.364204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.364205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.364256 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.364544 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.364545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.364580 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.364646 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.364741 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":57,"thread_start_us":70,"threads_started":1}
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996625, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.126@KRBTEST.COM for oryx/127.17.188.124@KRBTEST.COM
I20260620 23:03:46.373003 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.366936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39480 (local address 127.17.188.124:38545)
0620 23:03:46.366969 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.366971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.366978 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:03:46.367025 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.367026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.367033 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.367078 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.367328 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.367757 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.368413 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.368611 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.368681 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.370597 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.370598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.370598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.370603 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.371940 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.372363 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.372364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.372364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.372419 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.372613 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.372613 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.372614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.372705 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.372756 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.372911 (+ 155us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.372957 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:03:46.373071 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.366887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38545 (local address 127.0.0.1:39480)
0620 23:03:46.366925 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.366934 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.366967 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.367080 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.367080 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.367215 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.367250 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.367251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.367841 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.367842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.368335 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.368336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.368680 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.368681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.368745 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.369120 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.369124 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.370523 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.372043 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.372044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.372044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.372260 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.372486 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.372486 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.372487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.372551 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.372764 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.372764 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.372822 (+ 58us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.372895 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.372982 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260620 23:03:46.373562 24261 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.188.124:38545: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1"
member_type: NON_VOTER
last_known_addr {
host: "127.17.188.124"
port: 38545
}
attrs {
promote: true
}
}
dest_uuid: "033c16c578134bf284f736cec5956df6"
cas_config_opid_index: 5
I20260620 23:03:46.373868 24261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:03:46.374140 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 24323
I20260620 23:03:46.374189 24819 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1b0cd1229f0b4b94811801f7b83342b1 (127.17.188.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.375099 24819 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.188.124:38545
I20260620 23:03:46.375171 24819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "033c16c578134bf284f736cec5956df6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.375254 24819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:46.375593 24912 master.cc:537] Master@127.17.188.124:38545 shutting down...
W20260620 23:03:46.376058 24248 consensus_peers.cc:487] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 -> Peer 1b0cd1229f0b4b94811801f7b83342b1 (127.17.188.124:38545): Couldn't send request to peer 1b0cd1229f0b4b94811801f7b83342b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 23:03:46.381924 24912 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:46.381929 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:34825
--local_ip_for_outbound_sockets=127.17.188.65
--tserver_master_addrs=127.17.188.126:42405
--webserver_port=33901
--webserver_interface=127.17.188.65
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab
--principal=oryx/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.188.126:42405,127.17.188.125:36513
--tserver_master_addrs=127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:03:46.382004 24912 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:46.382045 24912 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: stopping tablet replica
I20260620 23:03:46.384853 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.375654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:36513 (local address 127.0.0.1:35130)
0620 23:03:46.381563 (+ 5909us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.381574 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.381621 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.381810 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.381811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.381812 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:46.381847 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.381849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.382725 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.382740 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.383949 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.383953 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.384580 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.384582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.384651 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.384673 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.384753 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.384787 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5896}
I20260620 23:03:46.384913 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.375840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35130 (local address 127.17.188.125:36513)
0620 23:03:46.381487 (+ 5647us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.381489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.381614 (+ 125us) server_negotiation.cc:407] Connection header received
0620 23:03:46.381670 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.381671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.381693 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.381758 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:46.381919 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.382634 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.384077 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.384498 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.384556 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.384598 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.384780 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.384822 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5617}
I20260620 23:03:46.385263 24868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 LEARNER]: Refusing update from remote peer 033c16c578134bf284f736cec5956df6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 23:03:46.385579 24820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 23:03:46.386277 24868 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1b0cd1229f0b4b94811801f7b83342b1 (127.17.188.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.387511 24897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 033c16c578134bf284f736cec5956df6. Latest consensus state: current_term: 1 leader_uuid: "033c16c578134bf284f736cec5956df6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.387611 24897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: This master's current role is: LEARNER
I20260620 23:03:46.387651 24897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "033c16c578134bf284f736cec5956df6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: NON_VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: true } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.387720 24897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: This master's current role is: LEARNER
I20260620 23:03:46.387964 24820 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6: attempting to promote NON_VOTER 7bddbce09fac4a67a49021ec992db0a7 to VOTER
I20260620 23:03:46.388152 24820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:03:46.388624 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.388504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38545 (local address 127.0.0.1:39494)
0620 23:03:46.388564 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.388599 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 23:03:46.388773 24868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 LEARNER]: Refusing update from remote peer 033c16c578134bf284f736cec5956df6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 23:03:46.389062 25008 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 23:03:46.389125 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.389012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38545 (local address 127.0.0.1:39500)
0620 23:03:46.389064 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.389095 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260620 23:03:46.389199 24248 consensus_peers.cc:487] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 -> Peer 1b0cd1229f0b4b94811801f7b83342b1 (127.17.188.124:38545): Couldn't send request to peer 1b0cd1229f0b4b94811801f7b83342b1. Status: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 23:03:46.389823 24820 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7bddbce09fac4a67a49021ec992db0a7 (127.17.188.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.390868 24819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "033c16c578134bf284f736cec5956df6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.390960 24819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:46.391256 24868 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7bddbce09fac4a67a49021ec992db0a7 (127.17.188.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.392132 25008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "033c16c578134bf284f736cec5956df6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.392210 25008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:46.392328 25007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "033c16c578134bf284f736cec5956df6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.392470 25007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:03:46.395036 24912 master.cc:552] Master@127.17.188.124:38545 shutdown complete.
I20260620 23:03:46.395085 24912 master_runner.cc:305] Clearing existing system tablet
I20260620 23:03:46.395213 24912 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:46.395258 24912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:46.396020 24912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:46.396554 25011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.396696 24912 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:46.396756 24912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.396781 24912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal
uuid: "1b0cd1229f0b4b94811801f7b83342b1"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "3fb60f0180c98fd4bf3e823269a7caaa"
server_key_iv: "df9ce44b2a4b4a17e4636b1d21f7d0a2"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.397575 24912 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:46.398851 24912 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 23:03:46.398931 24912 log.cc:1196] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 23:03:46.399104 24912 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: Deleting consensus metadata
I20260620 23:03:46.399358 24912 master_runner.cc:315] Copying system tablet from 127.17.188.126:42405
I20260620 23:03:46.399927 24912 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: tablet copy: Beginning tablet copy session from remote peer at address 127.17.188.126:42405
I20260620 23:03:46.407064 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.400249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33330 (local address 127.17.188.126:42405)
0620 23:03:46.400288 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.400290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.400352 (+ 62us) server_negotiation.cc:407] Connection header received
0620 23:03:46.400413 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.400414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.400424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.400494 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.400828 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.401760 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.402786 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.403082 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.403152 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.404179 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.404180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.404180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.404187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.405835 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.406274 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.406276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.406277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.406350 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.406542 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.406543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.406543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.406669 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.406735 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.406896 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.406976 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:03:46.407361 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.400187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:33330)
0620 23:03:46.400302 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.400312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.400365 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.400499 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.400500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.400685 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.400754 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.400755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.401854 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.401855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.402704 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.402707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.403160 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.403161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.403238 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.403726 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.403731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.404086 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.405964 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.405965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.405965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.406196 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.406423 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.406424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.406424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.406488 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.406739 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.406740 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.406785 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.406854 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.406946 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260620 23:03:46.407708 24281 tablet_copy_service.cc:140] P 033c16c578134bf284f736cec5956df6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1b0cd1229f0b4b94811801f7b83342b1 ({username='oryx', principal='oryx/127.17.188.124@KRBTEST.COM'} at 127.0.0.1:33330)
I20260620 23:03:46.407757 24281 tablet_copy_service.cc:161] P 033c16c578134bf284f736cec5956df6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1b0cd1229f0b4b94811801f7b83342b1 at {username='oryx', principal='oryx/127.17.188.124@KRBTEST.COM'} at 127.0.0.1:33330: session id = 1b0cd1229f0b4b94811801f7b83342b1-00000000000000000000000000000000
I20260620 23:03:46.408226 24281 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 23:03:46.408768 24912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 23:03:46.409865 24912 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: tablet copy: Starting download of 0 data blocks...
I20260620 23:03:46.410017 24912 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: tablet copy: Starting download of 1 WAL segments...
I20260620 23:03:46.412945 24912 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 23:03:46.413789 24281 tablet_copy_service.cc:342] P 033c16c578134bf284f736cec5956df6: Request end of tablet copy session 1b0cd1229f0b4b94811801f7b83342b1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.188.124@KRBTEST.COM'} at 127.0.0.1:33330
I20260620 23:03:46.413853 24281 tablet_copy_service.cc:433] P 033c16c578134bf284f736cec5956df6: ending tablet copy session 1b0cd1229f0b4b94811801f7b83342b1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1b0cd1229f0b4b94811801f7b83342b1
I20260620 23:03:46.416106 24912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:46.417212 25019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.417234 25018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.422281 25021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:46.422473 24912 server_base.cc:1034] running on GCE node
I20260620 23:03:46.422708 24912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:46.422837 24912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:46.423961 24912 hybrid_clock.cc:648] HybridClock initialized: now 1781996626423952 us; error 25 us; skew 500 ppm
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:46.425393 24912 init.cc:377] Logged in from keytab as oryx/127.17.188.124@KRBTEST.COM (short username oryx)
I20260620 23:03:46.425642 24912 webserver.cc:460] Webserver started at http://127.17.188.124:36447/ using document root <none> and password file <none>
I20260620 23:03:46.425753 24912 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:46.425793 24912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:46.426443 24912 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:46.426996 25028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.427153 24912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:03:46.427206 24912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal
uuid: "1b0cd1229f0b4b94811801f7b83342b1"
format_stamp: "Formatted at 2026-06-20 23:03:46 on dist-test-slave-zpss"
server_key: "3fb60f0180c98fd4bf3e823269a7caaa"
server_key_iv: "df9ce44b2a4b4a17e4636b1d21f7d0a2"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.427258 24912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:46.432775 24912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:46.434634 24912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:46.437565 24912 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.124:38545
I20260620 23:03:46.437872 24912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
I20260620 23:03:46.438109 25080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.124:38545 every 8 connection(s)
I20260620 23:03:46.438642 25081 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:03:46.439054 25081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: Bootstrap starting.
I20260620 23:03:46.441679 25081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:03:46.441879 25081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: Bootstrap complete.
I20260620 23:03:46.442034 25081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:03:46.442106 25081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b0cd1229f0b4b94811801f7b83342b1, State: Initialized, Role: LEARNER
I20260620 23:03:46.442152 25081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:03:46.442307 25081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:46.443446 25084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } } }
I20260620 23:03:46.443686 25084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1 [sys.catalog]: This master's current role is: LEARNER
I20260620 23:03:46.444453 25095 catalog_manager.cc:1259] Loaded cluster ID: e7790438fd30452f81920bc2452a953d
I20260620 23:03:46.444504 25095 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: loading cluster ID for follower catalog manager: success
I20260620 23:03:46.445498 25095 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: acquiring CA information for follower catalog manager: success
I20260620 23:03:46.445765 25095 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b0cd1229f0b4b94811801f7b83342b1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 23:03:46.475382 25005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:46.475608 25005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:46.475644 25005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:46.476704 25005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:46.476768 25005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:46.476810 25005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:03:46.478165 25005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab
--principal=oryx/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:34825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=33901
--tserver_master_addrs=127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:46.478396 25005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:46.478583 25005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:46.484493 25101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:46.484596 25005 server_base.cc:1034] running on GCE node
W20260620 23:03:46.484519 25100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.484748 25103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:46.484951 25005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:46.485177 25005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:46.486335 25005 hybrid_clock.cc:648] HybridClock initialized: now 1781996626486319 us; error 32 us; skew 500 ppm
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:46.488692 25005 init.cc:377] Logged in from keytab as oryx/127.17.188.65@KRBTEST.COM (short username oryx)
I20260620 23:03:46.489101 25005 webserver.cc:460] Webserver started at http://127.17.188.65:33901/ using document root <none> and password file <none>
I20260620 23:03:46.489302 25005 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:46.489382 25005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:46.490872 25005 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:46.491550 25110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.491745 25005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:03:46.491827 25005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "c4aa0a99765345199223664ed6baaaae"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "5731119056da26b464cf35a8b3cc7c6b"
server_key_iv: "1fc718b346d3739c9ff4286fbc3dbd48"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.492146 25005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:46.514140 25005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:46.515794 25005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:46.515929 25005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:46.516166 25005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:46.516443 25005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:46.516474 25005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.516513 25005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:46.516538 25005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.523167 25005 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:34825
I20260620 23:03:46.523234 25223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:34825 every 8 connection(s)
I20260620 23:03:46.523561 25005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:46.528092 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 25005
I20260620 23:03:46.528223 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 24457
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.125@KRBTEST.COM
I20260620 23:03:46.534219 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.524504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:52879 (local address 127.17.188.126:42405)
0620 23:03:46.524555 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.524556 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.524643 (+ 87us) server_negotiation.cc:407] Connection header received
0620 23:03:46.525058 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.525060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.525074 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.525138 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.526030 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.526736 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.528076 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.528371 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.528410 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.531380 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.531382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.531382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.531389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.532792 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.533168 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.533170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.533170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.533251 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.533529 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.533529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.533530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.533661 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.533704 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.534054 (+ 350us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.534131 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 23:03:46.534480 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.524658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:48531 (local address 127.17.188.125:36513)
0620 23:03:46.524711 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.524712 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.525167 (+ 455us) server_negotiation.cc:407] Connection header received
0620 23:03:46.525244 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.525246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.525269 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.525421 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.525976 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.526730 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.528864 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.529169 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.529260 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.531588 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.531589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.531590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.531598 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.533179 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.533519 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.533520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.533521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.533582 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.533766 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.533767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.533767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.533963 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.534034 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.534246 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.534335 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 23:03:46.534552 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.524436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:52879)
0620 23:03:46.524570 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.524579 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.524967 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.525200 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.525204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.525727 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.525885 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.525890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.526866 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.526867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.527986 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.527989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.528891 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.528891 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.528992 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.529538 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.529546 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.531298 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.532884 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.532890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.532892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.533087 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.533339 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.533340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.533340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.533451 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.533748 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.533754 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.533903 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.534018 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.534161 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":140,"thread_start_us":73,"threads_started":1}
I20260620 23:03:46.534860 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.524587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:36513 (local address 127.17.188.65:48531)
0620 23:03:46.525117 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.525125 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.525189 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.525435 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.525436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.525712 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.525884 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.525888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.526881 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.526882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.528801 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.528803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.529265 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.529266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.529322 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.530129 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.530134 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.531499 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.533248 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.533248 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.533249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.533451 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.533652 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.533652 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.533653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.533702 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.534065 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.534066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.534122 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.534606 (+ 484us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.534726 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":129,"thread_start_us":49,"threads_started":1}
I20260620 23:03:46.535034 25225 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42405
I20260620 23:03:46.535109 25225 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:46.535245 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:45627
--local_ip_for_outbound_sockets=127.17.188.66
--tserver_master_addrs=127.17.188.126:42405
--webserver_port=36213
--webserver_interface=127.17.188.66
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab
--principal=oryx/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.188.126:42405,127.17.188.125:36513
--tserver_master_addrs=127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:03:46.535293 25225 heartbeater.cc:511] Master 127.17.188.126:42405 requested a full tablet report, sending...
I20260620 23:03:46.535642 24261 ts_manager.cc:194] Re-registered known tserver with Master: c4aa0a99765345199223664ed6baaaae (127.17.188.65:34825)
I20260620 23:03:46.536092 24261 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:52879
I20260620 23:03:46.536765 25226 heartbeater.cc:347] Connected to a master server at 127.17.188.125:36513
I20260620 23:03:46.536823 25226 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:46.536904 25226 heartbeater.cc:511] Master 127.17.188.125:36513 requested a full tablet report, sending...
I20260620 23:03:46.537289 24858 ts_manager.cc:194] Registered new tserver with Master: c4aa0a99765345199223664ed6baaaae (127.17.188.65:34825)
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.124@KRBTEST.COM
I20260620 23:03:46.540478 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.524200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:41617 (local address 127.17.188.124:38545)
0620 23:03:46.524326 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.524330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.524418 (+ 88us) server_negotiation.cc:407] Connection header received
0620 23:03:46.525066 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.525067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.525080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.525477 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.525996 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.526821 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.528052 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.528253 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.535303 (+ 7050us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.538030 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.538031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.538031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.538037 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.539377 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.539783 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.539784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.539784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.539832 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.540050 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.540051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.540051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.540140 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.540184 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.540361 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.540419 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260620 23:03:46.540604 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.524159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38545 (local address 127.17.188.65:41617)
0620 23:03:46.524306 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.524353 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.524958 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.525200 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.525204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.525722 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.525885 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.525890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.526899 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.526900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.527959 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.527962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.535069 (+ 7107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.535070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.535157 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.536316 (+ 1159us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.536321 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.537954 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.539453 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.539454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.539455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.539696 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.539900 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.539901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.539902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.539976 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.540202 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.540203 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.540250 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.540349 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.540466 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":75,"thread_start_us":88,"threads_started":1}
I20260620 23:03:46.541002 25224 heartbeater.cc:347] Connected to a master server at 127.17.188.124:38545
I20260620 23:03:46.541069 25224 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:46.541121 25224 heartbeater.cc:511] Master 127.17.188.124:38545 requested a full tablet report, sending...
I20260620 23:03:46.541510 25045 ts_manager.cc:194] Registered new tserver with Master: c4aa0a99765345199223664ed6baaaae (127.17.188.65:34825)
W20260620 23:03:46.616547 25232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:46.616734 25232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:46.616753 25232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:46.617969 25232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:46.618010 25232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:46.618062 25232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:03:46.619789 25232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab
--principal=oryx/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:45627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=36213
--tserver_master_addrs=127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:46.620041 25232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:46.620332 25232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:46.625154 25240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.625178 25237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.625394 25238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:46.625537 25232 server_base.cc:1034] running on GCE node
I20260620 23:03:46.625680 25232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:46.625845 25232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:46.627007 25232 hybrid_clock.cc:648] HybridClock initialized: now 1781996626626992 us; error 30 us; skew 500 ppm
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:46.629379 25232 init.cc:377] Logged in from keytab as oryx/127.17.188.66@KRBTEST.COM (short username oryx)
I20260620 23:03:46.629767 25232 webserver.cc:460] Webserver started at http://127.17.188.66:36213/ using document root <none> and password file <none>
I20260620 23:03:46.629989 25232 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:46.630084 25232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:46.631350 25232 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:46.631866 25247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.632038 25232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.632128 25232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "54527a31bff649169685d8a02bfe7ac4"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "3a6a28793739feb4225b41ffda2bf892"
server_key_iv: "bbb4aa520610fdc351f28d484e9ce82a"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.632422 25232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:46.660470 25232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:46.662202 25232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:46.662343 25232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:46.662554 25232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:46.662822 25232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:46.662870 25232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.662909 25232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:46.662933 25232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.668363 25232 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:45627
I20260620 23:03:46.668428 25360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:45627 every 8 connection(s)
I20260620 23:03:46.668722 25232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:46.670920 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 25232
I20260620 23:03:46.671078 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 24591
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.124@KRBTEST.COM
I20260620 23:03:46.676388 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:33855
--local_ip_for_outbound_sockets=127.17.188.67
--tserver_master_addrs=127.17.188.126:42405
--webserver_port=44017
--webserver_interface=127.17.188.67
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab
--principal=oryx/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.188.126:42405,127.17.188.125:36513
--tserver_master_addrs=127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.125@KRBTEST.COM
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:03:46.680606 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.669404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:54867 (local address 127.17.188.124:38545)
0620 23:03:46.669460 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.669461 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.669793 (+ 332us) server_negotiation.cc:407] Connection header received
0620 23:03:46.670341 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.670343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.670358 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.670408 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.671652 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.672300 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.673664 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.674017 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.674061 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.677270 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.677271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.677272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.677279 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.678950 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.679510 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.679512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.679512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.679585 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.679885 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.679886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.679886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.680030 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.680071 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.680432 (+ 361us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.680519 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:03:46.680639 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.669987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:46677 (local address 127.17.188.125:36513)
0620 23:03:46.670055 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.670057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.670083 (+ 26us) server_negotiation.cc:407] Connection header received
0620 23:03:46.670356 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.670357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.670367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.670401 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.672792 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.673522 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.674522 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.674797 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.674862 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.677281 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.677282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.677282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.677288 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.679093 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.679603 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.679604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.679604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.679658 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.679885 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.679886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.679886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.680012 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.680059 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.680440 (+ 381us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.680526 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:03:46.680994 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.669429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38545 (local address 127.17.188.66:54867)
0620 23:03:46.669587 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.669626 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.670230 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.670469 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.670473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.670835 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.671555 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.671558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.672443 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.672444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.673573 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.673575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.674142 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.674142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.674286 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.674812 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.674822 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.677167 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.679040 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.679046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.679048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.679424 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.679658 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.679659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.679659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.679807 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.680266 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.680267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.680298 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.680726 (+ 428us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.680838 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260620 23:03:46.681021 25367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.669824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:36513 (local address 127.17.188.66:46677)
0620 23:03:46.669913 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.669921 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.670247 (+ 326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.670848 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.670849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.670979 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.671123 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.671129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.673615 (+ 2486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.673615 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.674457 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.674459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.674913 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.674914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.674979 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.675381 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.675386 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.677168 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.679167 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.679168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.679168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.679456 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.679727 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.679727 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.679727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.679807 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.680110 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.680115 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.680272 (+ 157us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.680407 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.680549 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":1,"thread_start_us":55,"threads_started":1}
I20260620 23:03:46.681684 25361 heartbeater.cc:347] Connected to a master server at 127.17.188.124:38545
I20260620 23:03:46.681746 25363 heartbeater.cc:347] Connected to a master server at 127.17.188.125:36513
I20260620 23:03:46.681785 25363 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:46.681999 25363 heartbeater.cc:511] Master 127.17.188.125:36513 requested a full tablet report, sending...
I20260620 23:03:46.681775 25361 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:46.682225 25361 heartbeater.cc:511] Master 127.17.188.124:38545 requested a full tablet report, sending...
I20260620 23:03:46.682410 24858 ts_manager.cc:194] Registered new tserver with Master: 54527a31bff649169685d8a02bfe7ac4 (127.17.188.66:45627)
I20260620 23:03:46.682802 25045 ts_manager.cc:194] Registered new tserver with Master: 54527a31bff649169685d8a02bfe7ac4 (127.17.188.66:45627)
I20260620 23:03:46.682932 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.669752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:51663 (local address 127.17.188.126:42405)
0620 23:03:46.669828 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.669830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.669837 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:03:46.670367 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.670369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.670380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.670411 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.673929 (+ 3518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.675055 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.676348 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.676648 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.676709 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.679954 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.679956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.679956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.679963 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.681761 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.682111 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.682113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.682113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.682168 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.682362 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.682363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.682364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.682515 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.682582 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.682721 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.682805 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 23:03:46.683063 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.669665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:51663)
0620 23:03:46.669769 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.669778 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.670247 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.670493 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.670494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.670837 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.671123 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.671129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.675404 (+ 4275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.675405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.676259 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.676261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.677038 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.677039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.677105 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.677920 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.677925 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.679855 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.681836 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.681836 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.681837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.682016 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.682237 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.682238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.682239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.682304 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.682582 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.682583 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.682624 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.682692 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.682780 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":4,"thread_start_us":56,"threads_started":1}
I20260620 23:03:46.683487 25362 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42405
I20260620 23:03:46.683544 25362 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:46.683612 25362 heartbeater.cc:511] Master 127.17.188.126:42405 requested a full tablet report, sending...
I20260620 23:03:46.683815 24261 ts_manager.cc:194] Re-registered known tserver with Master: 54527a31bff649169685d8a02bfe7ac4 (127.17.188.66:45627)
I20260620 23:03:46.684165 24261 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:51663
W20260620 23:03:46.758265 25368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:46.758445 25368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:46.758467 25368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:46.759593 25368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:46.759634 25368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:46.759685 25368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:03:46.761117 25368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab
--principal=oryx/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:33855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=44017
--tserver_master_addrs=127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:03:46.761372 25368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:46.761590 25368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:46.766186 25376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.766201 25374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:46.766201 25373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:46.766463 25368 server_base.cc:1034] running on GCE node
I20260620 23:03:46.766700 25368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:46.766944 25368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:46.768106 25368 hybrid_clock.cc:648] HybridClock initialized: now 1781996626768086 us; error 34 us; skew 500 ppm
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:46.770700 25368 init.cc:377] Logged in from keytab as oryx/127.17.188.67@KRBTEST.COM (short username oryx)
I20260620 23:03:46.771121 25368 webserver.cc:460] Webserver started at http://127.17.188.67:44017/ using document root <none> and password file <none>
I20260620 23:03:46.771309 25368 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:46.771358 25368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:46.772480 25368 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:03:46.772976 25383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.773187 25368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.773275 25368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "4d89b4354fb740cc84e8ff02e6f0b383"
format_stamp: "Formatted at 2026-06-20 23:03:45 on dist-test-slave-zpss"
server_key: "f81a4c1f046bbd24c8862833cd2542c0"
server_key_iv: "246cf51892cf8277fbeeb007df3f67a3"
server_key_version: "encryptionkey@0"
I20260620 23:03:46.773687 25368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:46.787714 25368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:46.789866 25368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:46.790004 25368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:46.790220 25368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:46.790504 25368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:46.790536 25368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.790557 25368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:46.790601 25368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:46.796128 25368 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:33855
I20260620 23:03:46.796180 25496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:33855 every 8 connection(s)
I20260620 23:03:46.796473 25368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:46.801920 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 25368
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.125@KRBTEST.COM
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
Jun 20 23:03:46 dist-test-slave-zpss krb5kdc[24216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.124@KRBTEST.COM
I20260620 23:03:46.808998 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.797392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:36513 (local address 127.17.188.67:52053)
0620 23:03:46.797496 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.797506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.797929 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.798335 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.798336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.798631 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.798706 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.798708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.799421 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.799422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.800520 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.800522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.800921 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.800922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.801034 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.801748 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.801758 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.804582 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.807211 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.807218 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.807219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.807532 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.807823 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.807824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.807824 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.807963 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.808301 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.808307 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.808471 (+ 164us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.808603 (+ 132us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.808761 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":62,"threads_started":1}
I20260620 23:03:46.809381 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.797450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:52053 (local address 127.17.188.125:36513)
0620 23:03:46.797493 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.797495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.797556 (+ 61us) server_negotiation.cc:407] Connection header received
0620 23:03:46.798242 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.798243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.798256 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.798306 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.798814 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.799320 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.800620 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.800826 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.800914 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.804705 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.804707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.804707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.804714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.807127 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.807648 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.807649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.807650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.807711 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.808051 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.808052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.808053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.808195 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.808268 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.809151 (+ 883us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.809231 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:03:46.809758 25499 heartbeater.cc:347] Connected to a master server at 127.17.188.125:36513
I20260620 23:03:46.809885 25499 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:46.810079 25499 heartbeater.cc:511] Master 127.17.188.125:36513 requested a full tablet report, sending...
I20260620 23:03:46.810412 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.797115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:39891)
0620 23:03:46.797271 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.797302 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.797929 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.798167 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.798177 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.798730 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.798799 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.798800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.799667 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.799668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.800348 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.800350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.800968 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.800969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.801041 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.801751 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.801764 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.804582 (+ 2818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.807941 (+ 3359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.807942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.807943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.808164 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.808385 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.808386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.808387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.808446 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.808719 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.808720 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.808773 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.808842 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.810281 (+ 1439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":23,"thread_start_us":77,"threads_started":1}
I20260620 23:03:46.810572 24858 ts_manager.cc:194] Registered new tserver with Master: 4d89b4354fb740cc84e8ff02e6f0b383 (127.17.188.67:33855)
I20260620 23:03:46.811465 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.797105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:39891 (local address 127.17.188.126:42405)
0620 23:03:46.797145 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.797147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.797404 (+ 257us) server_negotiation.cc:407] Connection header received
0620 23:03:46.798036 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.798037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.798049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.798105 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.798859 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.799587 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.800534 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.800834 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.800908 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.804695 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.804696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.804696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.804704 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.807331 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.808251 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.808253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.808253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.808313 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.808511 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.808512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.808513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.808635 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.808693 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.811288 (+ 2595us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.811355 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:03:46.811730 25498 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42405
I20260620 23:03:46.811774 25498 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:46.811829 25498 heartbeater.cc:511] Master 127.17.188.126:42405 requested a full tablet report, sending...
I20260620 23:03:46.812013 24261 ts_manager.cc:194] Re-registered known tserver with Master: 4d89b4354fb740cc84e8ff02e6f0b383 (127.17.188.67:33855)
I20260620 23:03:46.812472 24261 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:39891
I20260620 23:03:46.813490 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.803684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39516 (local address 127.17.188.124:38545)
0620 23:03:46.803888 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.803890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.804044 (+ 154us) server_negotiation.cc:407] Connection header received
0620 23:03:46.804152 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.804153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.804164 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.804219 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.804859 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.805603 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.806743 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.807049 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.807112 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.808696 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.808697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.808697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.808704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.810902 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.811307 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.811308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.811308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.811364 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.811555 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.811556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.811556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.812820 (+ 1264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.812912 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.813275 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.813377 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":53,"threads_started":1}
I20260620 23:03:46.813923 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.797604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:38649 (local address 127.17.188.124:38545)
0620 23:03:46.797644 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.797646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.797678 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:03:46.798102 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.798103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.798117 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.798187 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.798786 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.799554 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.800613 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.800913 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.800980 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.810869 (+ 9889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.810870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.810870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.810878 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.812431 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.812919 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.812920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.812921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.812981 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.813216 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.813217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.813217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.813380 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.813426 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.813687 (+ 261us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.813774 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:46.814095 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.802943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33342 (local address 127.17.188.126:42405)
0620 23:03:46.803075 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.803077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.803150 (+ 73us) server_negotiation.cc:407] Connection header received
0620 23:03:46.803227 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.803228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.803241 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.803305 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.805067 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.805857 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.806982 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.807312 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.807356 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.808865 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.808866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.808866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.808873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.811169 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.813143 (+ 1974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.813144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.813144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.813206 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.813441 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.813442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.813442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.813562 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.813632 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.813854 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.813957 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260620 23:03:46.814311 25503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.797542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38545 (local address 127.17.188.67:38649)
0620 23:03:46.797632 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.797640 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.797941 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.798193 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.798193 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.798527 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:46.798691 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.798695 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.799633 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.799634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.800522 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.800523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.801031 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.801032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.801086 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.801751 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:46.801764 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:46.810766 (+ 9002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:46.812547 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.812548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.812548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.812759 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.813062 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:46.813063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:46.813063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:46.813125 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:46.813481 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:46.813482 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:46.813525 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.814050 (+ 525us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.814167 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":33,"thread_start_us":61,"threads_started":1}
I20260620 23:03:46.814661 25497 heartbeater.cc:347] Connected to a master server at 127.17.188.124:38545
I20260620 23:03:46.814718 25497 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:46.814788 25497 heartbeater.cc:511] Master 127.17.188.124:38545 requested a full tablet report, sending...
I20260620 23:03:46.815063 25045 ts_manager.cc:194] Registered new tserver with Master: 4d89b4354fb740cc84e8ff02e6f0b383 (127.17.188.67:33855)
I20260620 23:03:46.817834 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.803260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.17.188.125:36513)
0620 23:03:46.803366 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.803367 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.803609 (+ 242us) server_negotiation.cc:407] Connection header received
0620 23:03:46.803709 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.803710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.803724 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.803778 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.804556 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.806225 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.812505 (+ 6280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.812811 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.812873 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.814855 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.814856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.814857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.814864 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.816541 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.816949 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.816950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.816950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.817006 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.817223 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.817223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.817224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.817383 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.817455 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.817669 (+ 214us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.817749 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260620 23:03:46.816246 24261 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:46.818742 24261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:46.826939 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.822852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:34825 (local address 127.0.0.1:43526)
0620 23:03:46.822889 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.822901 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.822949 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.823434 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.823435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.823438 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:46.823484 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.823486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.824541 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.824543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.825716 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.825720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.826710 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.826711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.826773 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.826777 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.826834 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.826873 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260620 23:03:46.827198 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.823002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43526 (local address 127.17.188.65:34825)
0620 23:03:46.823227 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.823232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.823244 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:03:46.823288 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.823290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.823311 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.823564 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:46.823631 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.824450 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.825804 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.826604 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.826707 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.826782 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.827053 (+ 271us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.827100 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":108,"threads_started":1}
I20260620 23:03:46.827888 25158 tablet_service.cc:1467] Processing CreateTablet for tablet d1f17f1b976d435383896f9413e1e0ea (DEFAULT_TABLE table=test-table [id=e36e4d07d1084aa8acc45da5f3877b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:46.828305 25158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1f17f1b976d435383896f9413e1e0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:46.828447 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.822877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:45627 (local address 127.0.0.1:43294)
0620 23:03:46.823034 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.823044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.823085 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.823956 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.823957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.823959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:46.823999 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.824000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.825387 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.825388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.826646 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.826650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.828162 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.828164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.828233 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.828238 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.828285 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.828331 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260620 23:03:46.829020 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.823566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.17.188.66:45627)
0620 23:03:46.823772 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.823777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.823787 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:03:46.823835 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.823837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.823852 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.824071 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:46.824130 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.825216 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.826785 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.828007 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.828804 (+ 797us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.828884 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.828910 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.828947 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260620 23:03:46.829535 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.823079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33855 (local address 127.0.0.1:59368)
0620 23:03:46.823191 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.823199 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.823230 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.824779 (+ 1549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.824780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.824782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:46.824816 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.824817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.827174 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.827175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.828449 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.828452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.829335 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.829336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.829394 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.829399 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.829434 (+ 35us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.829469 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":75,"threads_started":1}
I20260620 23:03:46.829654 25295 tablet_service.cc:1467] Processing CreateTablet for tablet d1f17f1b976d435383896f9413e1e0ea (DEFAULT_TABLE table=test-table [id=e36e4d07d1084aa8acc45da5f3877b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:46.829913 25295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1f17f1b976d435383896f9413e1e0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:46.830257 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.823340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59368 (local address 127.17.188.67:33855)
0620 23:03:46.824613 (+ 1273us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.824617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.824626 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:03:46.824660 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.824662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.824677 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.824884 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:46.824941 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.826988 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.828557 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.829204 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.829707 (+ 503us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.830118 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.830147 (+ 29us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.830183 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1243,"thread_start_us":89,"threads_started":1}
I20260620 23:03:46.830874 25431 tablet_service.cc:1467] Processing CreateTablet for tablet d1f17f1b976d435383896f9413e1e0ea (DEFAULT_TABLE table=test-table [id=e36e4d07d1084aa8acc45da5f3877b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:46.831221 25431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1f17f1b976d435383896f9413e1e0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:46.832158 25523 tablet_bootstrap.cc:492] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: Bootstrap starting.
I20260620 23:03:46.832854 25523 tablet_bootstrap.cc:654] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:46.832849 25524 tablet_bootstrap.cc:492] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: Bootstrap starting.
I20260620 23:03:46.833165 25523 log.cc:824] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:46.833477 25524 tablet_bootstrap.cc:654] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:46.833770 25524 log.cc:824] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:46.833979 25523 tablet_bootstrap.cc:492] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: No bootstrap required, opened a new log
I20260620 23:03:46.834059 25523 ts_tablet_manager.cc:1397] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:46.834417 25524 tablet_bootstrap.cc:492] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: No bootstrap required, opened a new log
I20260620 23:03:46.834497 25524 ts_tablet_manager.cc:1397] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:03:46.834776 25527 tablet_bootstrap.cc:492] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: Bootstrap starting.
I20260620 23:03:46.835428 25527 tablet_bootstrap.cc:654] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:46.835716 25527 log.cc:824] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:46.836010 25523 raft_consensus.cc:348] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } }
I20260620 23:03:46.836170 25523 raft_consensus.cc:374] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:46.836210 25523 raft_consensus.cc:729] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4aa0a99765345199223664ed6baaaae, State: Initialized, Role: FOLLOWER
I20260620 23:03:46.836334 25523 consensus_queue.cc:260] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } }
I20260620 23:03:46.836648 25523 ts_tablet_manager.cc:1428] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:46.836792 25225 heartbeater.cc:503] Master 127.17.188.126:42405 was elected leader, sending a full tablet report...
I20260620 23:03:46.837194 25524 raft_consensus.cc:348] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } }
I20260620 23:03:46.837415 25524 raft_consensus.cc:374] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:46.837459 25524 raft_consensus.cc:729] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54527a31bff649169685d8a02bfe7ac4, State: Initialized, Role: FOLLOWER
I20260620 23:03:46.837586 25524 consensus_queue.cc:260] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } }
I20260620 23:03:46.837838 25524 ts_tablet_manager.cc:1428] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:46.838011 25362 heartbeater.cc:503] Master 127.17.188.126:42405 was elected leader, sending a full tablet report...
I20260620 23:03:46.838472 25527 tablet_bootstrap.cc:492] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: No bootstrap required, opened a new log
I20260620 23:03:46.838536 25527 ts_tablet_manager.cc:1397] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 23:03:46.839991 25527 raft_consensus.cc:348] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } }
I20260620 23:03:46.840155 25527 raft_consensus.cc:374] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:46.840194 25527 raft_consensus.cc:729] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d89b4354fb740cc84e8ff02e6f0b383, State: Initialized, Role: FOLLOWER
I20260620 23:03:46.840291 25527 consensus_queue.cc:260] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } }
I20260620 23:03:46.840461 25527 ts_tablet_manager.cc:1428] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:03:46.840526 25498 heartbeater.cc:503] Master 127.17.188.126:42405 was elected leader, sending a full tablet report...
I20260620 23:03:46.846235 25530 raft_consensus.cc:482] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:46.846323 25530 raft_consensus.cc:504] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } }
I20260620 23:03:46.846604 25530 leader_election.cc:288] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4aa0a99765345199223664ed6baaaae (127.17.188.65:34825), 4d89b4354fb740cc84e8ff02e6f0b383 (127.17.188.67:33855)
I20260620 23:03:46.848979 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.846711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:34825 (local address 127.17.188.66:48305)
0620 23:03:46.846749 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.846760 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.846798 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.846922 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.846923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.846924 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:46.846988 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.846989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.847558 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.847559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.848242 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.848243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.848747 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.848748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.848826 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.848830 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.848887 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.848921 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260620 23:03:46.848976 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.846781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:48305 (local address 127.17.188.65:34825)
0620 23:03:46.846822 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.846836 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.846844 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:46.846863 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.846864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.846873 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.846920 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:46.847061 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.847481 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.848341 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.848662 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.848738 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.848745 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.848892 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.848922 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:03:46.849092 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.846916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:38619 (local address 127.17.188.67:33855)
0620 23:03:46.846949 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.846951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.846957 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:03:46.846974 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.846974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.846981 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.847046 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:46.847178 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.847595 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.848446 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.848784 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.848857 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.848863 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.848993 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.849022 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:03:46.849107 25367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.846750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33855 (local address 127.17.188.66:38619)
0620 23:03:46.846785 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:46.846795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:46.846831 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:46.847049 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:46.847050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:46.847052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:46.847109 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.847111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.847674 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.847674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.848357 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:46.848359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.848855 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.848856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:46.848926 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.848929 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:03:46.848990 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:03:46.849023 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260620 23:03:46.849574 25178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f17f1b976d435383896f9413e1e0ea" candidate_uuid: "54527a31bff649169685d8a02bfe7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4aa0a99765345199223664ed6baaaae" is_pre_election: true
I20260620 23:03:46.849707 25178 raft_consensus.cc:2454] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54527a31bff649169685d8a02bfe7ac4 in term 0.
I20260620 23:03:46.849934 25249 leader_election.cc:302] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54527a31bff649169685d8a02bfe7ac4, c4aa0a99765345199223664ed6baaaae; no voters:
I20260620 23:03:46.850111 25530 raft_consensus.cc:2781] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:46.850153 25530 raft_consensus.cc:482] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:46.850168 25530 raft_consensus.cc:3037] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:46.850237 25451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f17f1b976d435383896f9413e1e0ea" candidate_uuid: "54527a31bff649169685d8a02bfe7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" is_pre_election: true
I20260620 23:03:46.850363 25451 raft_consensus.cc:2454] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54527a31bff649169685d8a02bfe7ac4 in term 0.
I20260620 23:03:46.850841 25530 raft_consensus.cc:504] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } }
I20260620 23:03:46.850972 25530 leader_election.cc:288] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c4aa0a99765345199223664ed6baaaae (127.17.188.65:34825), 4d89b4354fb740cc84e8ff02e6f0b383 (127.17.188.67:33855)
I20260620 23:03:46.851090 25178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f17f1b976d435383896f9413e1e0ea" candidate_uuid: "54527a31bff649169685d8a02bfe7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4aa0a99765345199223664ed6baaaae"
I20260620 23:03:46.851163 25451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f17f1b976d435383896f9413e1e0ea" candidate_uuid: "54527a31bff649169685d8a02bfe7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d89b4354fb740cc84e8ff02e6f0b383"
I20260620 23:03:46.851166 25178 raft_consensus.cc:3037] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:46.851212 25451 raft_consensus.cc:3037] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:46.851940 25178 raft_consensus.cc:2454] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54527a31bff649169685d8a02bfe7ac4 in term 1.
I20260620 23:03:46.851940 25451 raft_consensus.cc:2454] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54527a31bff649169685d8a02bfe7ac4 in term 1.
I20260620 23:03:46.852113 25249 leader_election.cc:302] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54527a31bff649169685d8a02bfe7ac4, c4aa0a99765345199223664ed6baaaae; no voters:
I20260620 23:03:46.852252 25530 raft_consensus.cc:2781] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:46.852407 25530 raft_consensus.cc:686] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 1 LEADER]: Becoming Leader. State: Replica: 54527a31bff649169685d8a02bfe7ac4, State: Running, Role: LEADER
I20260620 23:03:46.852499 25530 consensus_queue.cc:237] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } }
I20260620 23:03:46.853263 24261 catalog_manager.cc:5495] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54527a31bff649169685d8a02bfe7ac4 (127.17.188.66). New cstate: current_term: 1 leader_uuid: "54527a31bff649169685d8a02bfe7ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54527a31bff649169685d8a02bfe7ac4" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 45627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:46.862994 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.860999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43308 (local address 127.17.188.66:45627)
0620 23:03:46.861065 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.861067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.861075 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:03:46.861102 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.861103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.861115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.861174 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:46.861290 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.861774 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.862424 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.862584 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.862628 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.862749 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:46.862838 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:46.862892 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.862930 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.862949 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:03:46.864570 25178 raft_consensus.cc:1261] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [term 1 FOLLOWER]: Refusing update from remote peer 54527a31bff649169685d8a02bfe7ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:46.864598 25451 raft_consensus.cc:1261] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [term 1 FOLLOWER]: Refusing update from remote peer 54527a31bff649169685d8a02bfe7ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:46.864862 25530 consensus_queue.cc:1035] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4aa0a99765345199223664ed6baaaae" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 34825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:46.865001 25532 consensus_queue.cc:1035] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d89b4354fb740cc84e8ff02e6f0b383" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:46.868031 25535 mvcc.cc:205] Tried to move back new op lower bound from 7299058183634796544 to 7299058183588335616. Current Snapshot: MvccSnapshot[applied={T|T < 7299058183634796544}]
I20260620 23:03:46.868309 25536 mvcc.cc:205] Tried to move back new op lower bound from 7299058183634796544 to 7299058183588335616. Current Snapshot: MvccSnapshot[applied={T|T < 7299058183634796544}]
I20260620 23:03:46.868808 25538 mvcc.cc:205] Tried to move back new op lower bound from 7299058183634796544 to 7299058183588335616. Current Snapshot: MvccSnapshot[applied={T|T < 7299058183634796544}]
I20260620 23:03:46.870733 24259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:46.870848 24259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:46.872001 24259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 033c16c578134bf284f736cec5956df6: Sending DeleteTablet for 3 replicas of tablet d1f17f1b976d435383896f9413e1e0ea
I20260620 23:03:46.872336 25158 tablet_service.cc:1514] Processing DeleteTablet for tablet d1f17f1b976d435383896f9413e1e0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:46 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43526
I20260620 23:03:46.872604 25545 tablet_replica.cc:330] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: stopping tablet replica
I20260620 23:03:46.872766 25545 raft_consensus.cc:2229] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:46.872768 25431 tablet_service.cc:1514] Processing DeleteTablet for tablet d1f17f1b976d435383896f9413e1e0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:46 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:59368
I20260620 23:03:46.872924 25295 tablet_service.cc:1514] Processing DeleteTablet for tablet d1f17f1b976d435383896f9413e1e0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:46 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43294
I20260620 23:03:46.872958 25545 raft_consensus.cc:2258] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:46.873389 25548 tablet_replica.cc:330] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: stopping tablet replica
I20260620 23:03:46.873476 25548 raft_consensus.cc:2229] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:46.873582 25548 raft_consensus.cc:2258] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:46.873598 25545 ts_tablet_manager.cc:1905] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:46.873713 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 24229
I20260620 23:03:46.875434 25545 ts_tablet_manager.cc:1918] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:46.875443 25548 ts_tablet_manager.cc:1905] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:46.875581 25545 log.cc:1196] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/d1f17f1b976d435383896f9413e1e0ea
I20260620 23:03:46.875949 25545 ts_tablet_manager.cc:1939] T d1f17f1b976d435383896f9413e1e0ea P c4aa0a99765345199223664ed6baaaae: Deleting consensus metadata
I20260620 23:03:46.879276 25548 ts_tablet_manager.cc:1918] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:46.879345 25548 log.cc:1196] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/d1f17f1b976d435383896f9413e1e0ea
I20260620 23:03:46.879559 25548 ts_tablet_manager.cc:1939] T d1f17f1b976d435383896f9413e1e0ea P 54527a31bff649169685d8a02bfe7ac4: Deleting consensus metadata
I20260620 23:03:46.879905 25547 tablet_replica.cc:330] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: stopping tablet replica
I20260620 23:03:46.880003 25547 raft_consensus.cc:2229] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:46.880117 25547 raft_consensus.cc:2258] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 23:03:46.880541 25249 connection.cc:535] server connection from 127.0.0.1:43294 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:03:46.880681 25547 ts_tablet_manager.cc:1905] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 23:03:46.880713 25111 connection.cc:535] server connection from 127.0.0.1:43526 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:03:46.882342 25547 ts_tablet_manager.cc:1918] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:46.882422 25547 log.cc:1196] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/d1f17f1b976d435383896f9413e1e0ea
I20260620 23:03:46.882690 25547 ts_tablet_manager.cc:1939] T d1f17f1b976d435383896f9413e1e0ea P 4d89b4354fb740cc84e8ff02e6f0b383: Deleting consensus metadata
W20260620 23:03:46.883003 25385 connection.cc:441] server connection from 127.0.0.1:59368 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59368 (request call id 1) could send its response
I20260620 23:03:46.883561 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.873086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39522 (local address 127.17.188.124:38545)
0620 23:03:46.873125 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.873127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.873296 (+ 169us) server_negotiation.cc:407] Connection header received
0620 23:03:46.873776 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.873777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.873788 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.873817 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.876252 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.877039 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.878156 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.878423 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.878489 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.880340 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.880341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.880341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.880349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.882150 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.882632 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.882633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.882634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.882692 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.882941 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.882942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.882942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.883061 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.883141 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.883398 (+ 257us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.883470 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:46.889739 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.882485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.17.188.125:36513)
0620 23:03:46.882530 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.882532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.883190 (+ 658us) server_negotiation.cc:407] Connection header received
0620 23:03:46.883252 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.883260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.883272 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.883341 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.883990 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.884601 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.885215 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.885516 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.885603 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.886903 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.886904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.886905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.886913 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.888504 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.888884 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.888885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.888886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.888942 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.889139 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.889140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.889140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.889260 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.889377 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.889558 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.889651 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:03:46.889876 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:46.882722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39534 (local address 127.17.188.124:38545)
0620 23:03:46.882769 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:46.882771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:46.883350 (+ 579us) server_negotiation.cc:407] Connection header received
0620 23:03:46.883438 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:46.883439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:46.883450 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:46.883480 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:46.884173 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.884917 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.885922 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:46.886165 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:46.886249 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:46.887319 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:46.887320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:46.887320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:46.887326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:46.888743 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.889067 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.889068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.889068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.889112 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:46.889303 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:46.889303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:46.889304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:46.889440 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:46.889528 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:46.889617 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 23:03:46.889696 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:03:47.869673 25556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:47.869345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:43015)
0620 23:03:47.869543 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:47.869617 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":62,"threads_started":1}
I20260620 23:03:47.869716 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:47.869412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:42105)
0620 23:03:47.869601 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:47.869667 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":54,"threads_started":1}
W20260620 23:03:47.869990 25362 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:42405 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:03:47.870081 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:47.869500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:58547)
0620 23:03:47.869978 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:47.870043 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":45,"threads_started":1}
W20260620 23:03:47.870213 25498 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:42405 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
W20260620 23:03:47.870292 25225 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:42405 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:03:48.372253 25559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 033c16c578134bf284f736cec5956df6)
I20260620 23:03:48.372418 25559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:03:48.372645 25559 leader_election.cc:288] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405)
I20260620 23:03:48.372964 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:48.372772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:33774)
0620 23:03:48.372883 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:48.372931 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
W20260620 23:03:48.373092 24845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:03:48.373500 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:48.373405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:33788)
0620 23:03:48.373451 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:48.373476 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:03:48.373595 24845 leader_election.cc:334] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405): Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:03:48.373643 24845 leader_election.cc:302] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7bddbce09fac4a67a49021ec992db0a7; no voters: 033c16c578134bf284f736cec5956df6
I20260620 23:03:48.373728 25559 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:03:48.870543 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:48.870296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:43955)
0620 23:03:48.870440 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:48.870486 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260620 23:03:48.870925 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:48.870655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:60547)
0620 23:03:48.870848 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:48.870883 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":50,"threads_started":1}
I20260620 23:03:48.871019 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:48.870786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:49859)
0620 23:03:48.870946 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:48.870978 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":58,"threads_started":1}
I20260620 23:03:49.871423 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.871038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:47617)
0620 23:03:49.871321 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:49.871365 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260620 23:03:49.871836 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.871497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:52359)
0620 23:03:49.871757 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:49.871794 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":77,"threads_started":1}
I20260620 23:03:49.871877 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.871404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:51729)
0620 23:03:49.871789 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:49.871828 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":59,"threads_started":1}
I20260620 23:03:50.272002 25568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 033c16c578134bf284f736cec5956df6)
I20260620 23:03:50.272140 25568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:03:50.272346 25568 leader_election.cc:288] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405)
I20260620 23:03:50.272727 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:50.272534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:34048)
0620 23:03:50.272655 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:50.272690 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
I20260620 23:03:50.273135 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:50.273048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:34064)
0620 23:03:50.273093 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:50.273119 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 23:03:50.273211 24845 leader_election.cc:334] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405): Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:03:50.273288 24845 leader_election.cc:302] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7bddbce09fac4a67a49021ec992db0a7; no voters: 033c16c578134bf284f736cec5956df6
I20260620 23:03:50.273391 25568 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:03:50.872479 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:50.871961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:51261)
0620 23:03:50.872375 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:50.872420 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":61,"threads_started":1}
I20260620 23:03:50.872637 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:50.872384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:33575)
0620 23:03:50.872553 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:50.872586 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":49,"threads_started":1}
I20260620 23:03:50.872710 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:50.872494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:33763)
0620 23:03:50.872636 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:50.872670 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":44,"threads_started":1}
I20260620 23:03:51.873313 25575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:51.873045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:40599)
0620 23:03:51.873183 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:51.873255 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260620 23:03:51.873649 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:51.873064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:51745)
0620 23:03:51.873576 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:51.873612 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":44,"threads_started":1}
I20260620 23:03:51.873734 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:51.873036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:34593)
0620 23:03:51.873615 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:51.873675 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":55,"threads_started":1}
I20260620 23:03:52.176889 25577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 033c16c578134bf284f736cec5956df6)
I20260620 23:03:52.176972 25577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:03:52.177078 25577 leader_election.cc:288] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405)
I20260620 23:03:52.177450 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.177256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:34276)
0620 23:03:52.177359 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.177416 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":64,"threads_started":1}
I20260620 23:03:52.177871 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.177793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:34288)
0620 23:03:52.177824 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.177847 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260620 23:03:52.177950 24845 leader_election.cc:334] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405): Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:03:52.177994 24845 leader_election.cc:302] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7bddbce09fac4a67a49021ec992db0a7; no voters: 033c16c578134bf284f736cec5956df6
I20260620 23:03:52.178061 25577 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:03:52.874411 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.874126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:53067)
0620 23:03:52.874314 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.874359 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":63,"threads_started":1}
I20260620 23:03:52.874598 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.874288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:45185)
0620 23:03:52.874512 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.874548 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":68,"threads_started":1}
I20260620 23:03:52.874724 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.874288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:46965)
0620 23:03:52.874641 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.874679 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":65,"threads_started":1}
I20260620 23:03:53.875335 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:53.875046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:47913)
0620 23:03:53.875235 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:53.875278 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":63,"threads_started":1}
I20260620 23:03:53.875411 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:53.875046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:57067)
0620 23:03:53.875316 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:53.875360 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":60,"threads_started":1}
I20260620 23:03:53.875697 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:53.875424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:58399)
0620 23:03:53.875620 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:53.875653 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260620 23:03:54.167163 25586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 033c16c578134bf284f736cec5956df6)
I20260620 23:03:54.167241 25586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:03:54.167361 25586 leader_election.cc:288] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405)
I20260620 23:03:54.167671 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:54.167518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:40592)
0620 23:03:54.167608 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:54.167637 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":44,"threads_started":1}
W20260620 23:03:54.167790 24845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260620 23:03:54.168144 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:54.168059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:40594)
0620 23:03:54.168103 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:54.168128 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:03:54.168262 24845 leader_election.cc:334] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405): Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:03:54.168318 24845 leader_election.cc:302] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7bddbce09fac4a67a49021ec992db0a7; no voters: 033c16c578134bf284f736cec5956df6
I20260620 23:03:54.168406 25586 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:03:54.876293 25589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:54.876057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:43603)
0620 23:03:54.876190 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:54.876233 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260620 23:03:54.876432 25590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:54.876085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:56039)
0620 23:03:54.876308 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:54.876341 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":37,"threads_started":1}
I20260620 23:03:54.876499 25591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:54.876229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:38155)
0620 23:03:54.876436 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:54.876466 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":61,"threads_started":1}
I20260620 23:03:55.877156 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:55.876868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:51681)
0620 23:03:55.877056 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:55.877099 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":74,"threads_started":1}
I20260620 23:03:55.877200 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:55.876867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:43223)
0620 23:03:55.877138 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:55.877166 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":80,"threads_started":1}
I20260620 23:03:55.877458 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:55.877067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:42159)
0620 23:03:55.877375 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:55.877410 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":60,"threads_started":1}
I20260620 23:03:56.877992 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.877773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:48089)
0620 23:03:56.877887 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:56.877935 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
I20260620 23:03:56.878031 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.877561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:56733)
0620 23:03:56.877962 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:56.877990 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":67,"threads_started":1}
I20260620 23:03:56.878139 25597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.877968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:47617)
0620 23:03:56.878058 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:56.878092 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":39,"threads_started":1}
I20260620 23:03:57.878742 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:57.878458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:49687)
0620 23:03:57.878683 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:57.878710 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":48,"threads_started":1}
I20260620 23:03:57.878733 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:57.878441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:59993)
0620 23:03:57.878638 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:57.878680 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":63,"threads_started":1}
I20260620 23:03:57.878912 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:57.878456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:49687)
0620 23:03:57.878742 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:57.878781 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":69,"threads_started":1}
I20260620 23:03:58.387632 25601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 033c16c578134bf284f736cec5956df6)
I20260620 23:03:58.387710 25601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:03:58.387859 25601 leader_election.cc:288] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405)
I20260620 23:03:58.388144 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:58.387983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:40892)
0620 23:03:58.388081 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:58.388109 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":51,"threads_started":1}
I20260620 23:03:58.388528 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:58.388441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:40896)
0620 23:03:58.388486 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:58.388511 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 23:03:58.388603 24845 leader_election.cc:334] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405): Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:03:58.388671 24845 leader_election.cc:302] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7bddbce09fac4a67a49021ec992db0a7; no voters: 033c16c578134bf284f736cec5956df6
I20260620 23:03:58.388758 25601 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:03:58.879509 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:58.879244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:33663)
0620 23:03:58.879399 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:58.879454 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":62,"threads_started":1}
I20260620 23:03:58.879563 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:58.879274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:59819)
0620 23:03:58.879464 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:58.879502 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":66,"threads_started":1}
I20260620 23:03:58.879580 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:58.879244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:56093)
0620 23:03:58.879496 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:58.879525 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":64,"threads_started":1}
I20260620 23:03:59.880431 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.880151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:59059)
0620 23:03:59.880311 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.880368 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":49,"threads_started":1}
I20260620 23:03:59.880431 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.879873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:43799)
0620 23:03:59.880325 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.880367 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":58,"threads_started":1}
I20260620 23:03:59.880509 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.880223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:43109)
0620 23:03:59.880444 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.880473 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":44,"threads_started":1}
I20260620 23:04:00.881243 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:00.880957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:48347)
0620 23:04:00.881130 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:00.881182 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":63,"threads_started":1}
I20260620 23:04:00.881407 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:00.881096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:53487)
0620 23:04:00.881277 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:00.881345 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":50,"threads_started":1}
I20260620 23:04:00.881516 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:00.881266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:43917)
0620 23:04:00.881441 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:00.881478 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
I20260620 23:04:01.882191 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:01.881940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:60123)
0620 23:04:01.882098 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:01.882140 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260620 23:04:01.882267 25615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:01.882031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:37027)
0620 23:04:01.882191 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:01.882224 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":39,"threads_started":1}
I20260620 23:04:01.882510 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:01.882001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:51927)
0620 23:04:01.882401 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:01.882468 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":51,"threads_started":1}
I20260620 23:04:02.883139 25617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:02.882871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:50231)
0620 23:04:02.883043 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:02.883087 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":58,"threads_started":1}
I20260620 23:04:02.883162 25616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:02.882846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:44129)
0620 23:04:02.883071 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:02.883114 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":66,"threads_started":1}
I20260620 23:04:02.883174 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:02.882931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:57185)
0620 23:04:02.883108 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:02.883140 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":45,"threads_started":1}
I20260620 23:04:03.883792 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.883526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:47467)
0620 23:04:03.883706 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.883745 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":61,"threads_started":1}
I20260620 23:04:03.883792 25621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.883636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:42593)
0620 23:04:03.883726 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.883750 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":41,"threads_started":1}
I20260620 23:04:03.883854 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.883559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:56701)
0620 23:04:03.883762 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.883802 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":53,"threads_started":1}
I20260620 23:04:04.698033 25622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 033c16c578134bf284f736cec5956df6)
I20260620 23:04:04.698109 25622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:04:04.698220 25622 leader_election.cc:288] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405)
I20260620 23:04:04.698527 25623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:04.698367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:35852)
0620 23:04:04.698457 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:04.698494 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":52,"threads_started":1}
W20260620 23:04:04.698601 24845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260620 23:04:04.698932 25623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:04.698849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:35862)
0620 23:04:04.698893 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:04.698918 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 23:04:04.699004 24845 leader_election.cc:334] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405): Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:04:04.699074 24845 leader_election.cc:302] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7bddbce09fac4a67a49021ec992db0a7; no voters: 033c16c578134bf284f736cec5956df6
I20260620 23:04:04.699162 25622 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:04:04.884616 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:04.884332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:56705)
0620 23:04:04.884520 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:04.884562 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":60,"threads_started":1}
I20260620 23:04:04.884680 25627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:04.884479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:40401)
0620 23:04:04.884593 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:04.884628 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260620 23:04:04.884950 25626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:04.884332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:47679)
0620 23:04:04.884851 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:04.884913 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":82,"threads_started":1}
I20260620 23:04:05.885448 25628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:05.885157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:59559)
0620 23:04:05.885358 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:05.885398 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":59,"threads_started":1}
I20260620 23:04:05.885574 25630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:05.885344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:59431)
0620 23:04:05.885494 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:05.885528 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260620 23:04:05.885659 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:05.885157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:55369)
0620 23:04:05.885560 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:05.885595 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":57,"threads_started":1}
I20260620 23:04:06.827725 25631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 033c16c578134bf284f736cec5956df6)
I20260620 23:04:06.827853 25631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "033c16c578134bf284f736cec5956df6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42405 } } peers { permanent_uuid: "7bddbce09fac4a67a49021ec992db0a7" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 36513 } attrs { promote: false } } peers { permanent_uuid: "1b0cd1229f0b4b94811801f7b83342b1" member_type: NON_VOTER last_known_addr { host: "127.17.188.124" port: 38545 } attrs { promote: true } }
I20260620 23:04:06.828053 25631 leader_election.cc:288] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405)
I20260620 23:04:06.828418 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.828238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:35950)
0620 23:04:06.828337 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.828381 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":55,"threads_started":1}
I20260620 23:04:06.828835 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.828730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.0.0.1:35954)
0620 23:04:06.828777 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.828802 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 23:04:06.828907 24845 leader_election.cc:334] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 033c16c578134bf284f736cec5956df6 (127.17.188.126:42405): Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
I20260620 23:04:06.828938 24845 leader_election.cc:302] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7bddbce09fac4a67a49021ec992db0a7; no voters: 033c16c578134bf284f736cec5956df6
I20260620 23:04:06.829020 25631 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7bddbce09fac4a67a49021ec992db0a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:04:06.886365 25634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.885959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:36225)
0620 23:04:06.886272 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.886316 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":64,"threads_started":1}
I20260620 23:04:06.886365 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.885960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:39199)
0620 23:04:06.886266 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.886316 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":67,"threads_started":1}
I20260620 23:04:06.886562 25636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.886112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:40477)
0620 23:04:06.886423 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.886513 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":47,"threads_started":1}
I20260620 23:04:07.887137 25637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:07.886829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:48317)
0620 23:04:07.887038 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:07.887078 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":83,"threads_started":1}
I20260620 23:04:07.887248 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:07.887074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:48911)
0620 23:04:07.887178 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:07.887206 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":42,"threads_started":1}
I20260620 23:04:07.887284 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:07.886967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:52527)
0620 23:04:07.887193 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:07.887230 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":49,"threads_started":1}
I20260620 23:04:08.887984 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:08.887654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:40211)
0620 23:04:08.887868 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:08.887927 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":93,"threads_started":1}
I20260620 23:04:08.888044 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:08.887738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:48929)
0620 23:04:08.887959 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:08.887996 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":55,"threads_started":1}
I20260620 23:04:08.888157 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:08.887955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:53937)
0620 23:04:08.888090 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:08.888121 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":41,"threads_started":1}
I20260620 23:04:09.888760 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:09.888534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:42847)
0620 23:04:09.888655 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:09.888699 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260620 23:04:09.888866 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:09.888575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:35859)
0620 23:04:09.888753 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:09.888792 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":65,"threads_started":1}
I20260620 23:04:09.888959 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:09.888534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:58317)
0620 23:04:09.888891 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:09.888924 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":59,"threads_started":1}
I20260620 23:04:10.889720 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:10.889499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:43623)
0620 23:04:10.889627 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:10.889665 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":53,"threads_started":1}
I20260620 23:04:10.889746 25646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:10.889486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:34133)
0620 23:04:10.889668 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:10.889703 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":61,"threads_started":1}
I20260620 23:04:10.889783 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:10.889498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:39397)
0620 23:04:10.889702 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:10.889734 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":55,"threads_started":1}
I20260620 23:04:11.890462 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:11.890121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:50383)
0620 23:04:11.890362 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:11.890407 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":56,"threads_started":1}
I20260620 23:04:11.890589 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:11.890335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:42963)
0620 23:04:11.890503 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:11.890540 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":68,"threads_started":1}
I20260620 23:04:11.890786 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:11.890459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:37059)
0620 23:04:11.890624 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:11.890653 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":44,"threads_started":1}
I20260620 23:04:12.891495 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:12.891262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:55339)
0620 23:04:12.891391 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:12.891436 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260620 23:04:12.891556 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:12.891321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:51185)
0620 23:04:12.891496 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:12.891522 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":43,"threads_started":1}
I20260620 23:04:12.891552 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:12.891174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:53139)
0620 23:04:12.891468 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:12.891508 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":69,"threads_started":1}
I20260620 23:04:13.892426 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:13.891888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:38703)
0620 23:04:13.892313 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:13.892362 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":89,"threads_started":1}
I20260620 23:04:13.892532 25656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:13.891957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:40427)
0620 23:04:13.892366 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:13.892480 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":71,"threads_started":1}
I20260620 23:04:13.892539 25657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:13.892201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:48803)
0620 23:04:13.892461 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:13.892501 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":47,"threads_started":1}
I20260620 23:04:14.893203 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:14.893026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:49893)
0620 23:04:14.893117 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:14.893151 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":40,"threads_started":1}
I20260620 23:04:14.893224 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:14.892999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:60653)
0620 23:04:14.893162 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:14.893187 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":65,"threads_started":1}
I20260620 23:04:14.893208 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:14.893013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:35579)
0620 23:04:14.893122 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:14.893157 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260620 23:04:15.893818 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:15.893540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:40179)
0620 23:04:15.893719 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:15.893761 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":57,"threads_started":1}
I20260620 23:04:15.893841 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:15.893602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:40105)
0620 23:04:15.893739 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:15.893777 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":49,"threads_started":1}
I20260620 23:04:15.893930 25663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:15.893676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:46731)
0620 23:04:15.893837 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:15.893891 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":45,"threads_started":1}
I20260620 23:04:16.894495 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:16.894278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.65:43171)
0620 23:04:16.894401 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:16.894442 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260620 23:04:16.894585 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:16.894334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.67:33333)
0620 23:04:16.894509 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:16.894543 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":40,"threads_started":1}
I20260620 23:04:16.894634 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:16.894379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42405 (local address 127.17.188.66:34903)
0620 23:04:16.894509 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:16.894543 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":50,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.17.188.126:42405,127.17.188.125:36513,127.17.188.124:38545, num_attempts: 1)
I20260620 23:04:17.061162 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 25005
I20260620 23:04:17.065857 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 25232
I20260620 23:04:17.070923 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 25368
I20260620 23:04:17.074972 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 24725
I20260620 23:04:17.079116 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 24912
2026-06-20T23:04:17Z chronyd exiting
I20260620 23:04:17.094517 18161 test_util.cc:175] -----------------------------------------------
I20260620 23:04:17.094553 18161 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996599092328-18161-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (32940 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:17 dist-test-slave-zpss krb5kdc[25669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:17 dist-test-slave-zpss krb5kdc[25669](info): set up 2 sockets
Jun 20 23:04:17 dist-test-slave-zpss krb5kdc[25669](info): commencing operation
krb5kdc: starting...
W20260620 23:04:18.118330 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:18Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.126.keytab.
Entry for principal oryx/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.126.keytab.
I20260620 23:04:18.232120 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34913
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.126.keytab
--principal=oryx/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:34913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:18.310249 25682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:18.310422 25682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:18.310443 25682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:18.311561 25682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:18.311600 25682 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 23:04:18.311614 25682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:18.311626 25682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:18.311638 25682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:18.311688 25682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:18.313211 25682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:34913
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.126.keytab
--principal=oryx/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:18.313465 25682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:18.313683 25682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:18.318174 25687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:18.318183 25688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:18.318184 25690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:18.318449 25682 server_base.cc:1034] running on GCE node
I20260620 23:04:18.318771 25682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:18.319052 25682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:18.320204 25682 hybrid_clock.cc:648] HybridClock initialized: now 1781996658320201 us; error 35 us; skew 500 ppm
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:18.322521 25682 init.cc:377] Logged in from keytab as oryx/127.17.188.126@KRBTEST.COM (short username oryx)
I20260620 23:04:18.322873 25682 webserver.cc:460] Webserver started at http://127.17.188.126:39015/ using document root <none> and password file <none>
I20260620 23:04:18.323063 25682 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:18.323101 25682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:18.323181 25682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:18.324092 25682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "a1619ed2cc7f4b87be05697f82d37943"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "6f2fcc426458ffac1ab2fa1e8f805875"
server_key_iv: "80472b56721b0330197995c9b63b3977"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.324378 25682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "a1619ed2cc7f4b87be05697f82d37943"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "6f2fcc426458ffac1ab2fa1e8f805875"
server_key_iv: "80472b56721b0330197995c9b63b3977"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.325598 25682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:04:18.326232 25697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.326395 25682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.326469 25682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "a1619ed2cc7f4b87be05697f82d37943"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "6f2fcc426458ffac1ab2fa1e8f805875"
server_key_iv: "80472b56721b0330197995c9b63b3977"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.326521 25682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:18.338224 25682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:18.340333 25682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:18.340446 25682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:18.344673 25682 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:34913
I20260620 23:04:18.344772 25759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:34913 every 8 connection(s)
I20260620 23:04:18.345045 25682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:18.345687 25760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:18.345896 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 25682
I20260620 23:04:18.346017 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:04:18.346194 18161 external_mini_cluster.cc:1428] Setting key 4505e6684e72d5863098d034a5aa725f
I20260620 23:04:18.348438 25760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943: Bootstrap starting.
I20260620 23:04:18.349112 25760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:18.349448 25760 log.cc:824] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:18.350104 25760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943: No bootstrap required, opened a new log
I20260620 23:04:18.352068 25760 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1619ed2cc7f4b87be05697f82d37943" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34913 } }
I20260620 23:04:18.352207 25760 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:18.352231 25760 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1619ed2cc7f4b87be05697f82d37943, State: Initialized, Role: FOLLOWER
I20260620 23:04:18.352367 25760 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1619ed2cc7f4b87be05697f82d37943" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34913 } }
I20260620 23:04:18.352447 25760 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:18.352478 25760 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:18.352516 25760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:04:18.353099 25760 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1619ed2cc7f4b87be05697f82d37943" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34913 } }
I20260620 23:04:18.353215 25760 leader_election.cc:302] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1619ed2cc7f4b87be05697f82d37943; no voters:
I20260620 23:04:18.353394 25760 leader_election.cc:288] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:18.353427 25765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:18.353538 25765 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [term 1 LEADER]: Becoming Leader. State: Replica: a1619ed2cc7f4b87be05697f82d37943, State: Running, Role: LEADER
I20260620 23:04:18.353613 25765 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1619ed2cc7f4b87be05697f82d37943" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34913 } }
I20260620 23:04:18.353641 25760 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:18.354122 25767 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1619ed2cc7f4b87be05697f82d37943. Latest consensus state: current_term: 1 leader_uuid: "a1619ed2cc7f4b87be05697f82d37943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1619ed2cc7f4b87be05697f82d37943" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34913 } } }
I20260620 23:04:18.354216 25767 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:18.354123 25766 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1619ed2cc7f4b87be05697f82d37943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1619ed2cc7f4b87be05697f82d37943" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34913 } } }
I20260620 23:04:18.354326 25766 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:18.354502 25772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:18.355002 25772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:18.356524 25772 catalog_manager.cc:1347] Generated new cluster ID: d5a3c5fd3b504afb90b48a938af435c3
I20260620 23:04:18.356567 25772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:18.358146 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.346692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.17.188.126:34913)
0620 23:04:18.346872 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.346881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.346911 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:04:18.347432 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.347440 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.347508 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.347642 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:18.348205 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.349556 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.350442 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.351103 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.351224 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.353564 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:18.353568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:18.353568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:18.353580 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:18.356473 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.356844 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.356854 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.356854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.356938 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.357133 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.357134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.357135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.357440 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:18.357586 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.357798 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.357927 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":59,"threads_started":1}
I20260620 23:04:18.376889 25772 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:18.377599 25772 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:18.386845 25772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943: Generated new TSK 0
I20260620 23:04:18.387001 25772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab.
Entry for principal oryx/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab.
I20260620 23:04:18.418143 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34913
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab
--principal=oryx/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:18.497936 25786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:18.498129 25786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:18.498162 25786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:18.499313 25786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:18.499351 25786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:18.499372 25786 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:04:18.499388 25786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:04:18.500842 25786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab
--principal=oryx/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.188.126:34913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:18.501060 25786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:18.501298 25786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:18.505913 25794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:18.505983 25792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:18.505915 25791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:18.505993 25786 server_base.cc:1034] running on GCE node
I20260620 23:04:18.506330 25786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:18.506552 25786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:18.507684 25786 hybrid_clock.cc:648] HybridClock initialized: now 1781996658507676 us; error 26 us; skew 500 ppm
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:18.514487 25786 init.cc:377] Logged in from keytab as oryx/127.17.188.65@KRBTEST.COM (short username oryx)
I20260620 23:04:18.514968 25786 webserver.cc:460] Webserver started at http://127.17.188.65:39779/ using document root <none> and password file <none>
I20260620 23:04:18.515178 25786 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:18.515239 25786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:18.515383 25786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:18.516387 25786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "da8933fda0cb410684faff151335307b"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "f351f19fcb9dd7eb161dc8b7b449383c"
server_key_iv: "6a7e28a9d3ccb750ce4b76466bf22f95"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.516736 25786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "da8933fda0cb410684faff151335307b"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "f351f19fcb9dd7eb161dc8b7b449383c"
server_key_iv: "6a7e28a9d3ccb750ce4b76466bf22f95"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.518122 25786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:04:18.518795 25801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.518949 25786 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:18.519040 25786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "da8933fda0cb410684faff151335307b"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "f351f19fcb9dd7eb161dc8b7b449383c"
server_key_iv: "6a7e28a9d3ccb750ce4b76466bf22f95"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.519114 25786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:18.527745 25786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:18.529423 25786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:18.529556 25786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:18.530090 25786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:18.530140 25786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.530205 25786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:18.530232 25786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.536441 25786 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:46681
I20260620 23:04:18.536517 25916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:46681 every 8 connection(s)
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:04:18.536764 25786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:04:18.539418 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.530394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:60689 (local address 127.17.188.126:34913)
0620 23:04:18.530443 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.530445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.530663 (+ 218us) server_negotiation.cc:407] Connection header received
0620 23:04:18.531067 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.531068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.531082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.531163 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:18.531854 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.532721 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.534336 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.534695 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.534761 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.537221 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:18.537223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:18.537223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:18.537229 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:18.538407 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.538734 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.538735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.538735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.538784 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.538956 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.538957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.538957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.539047 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:18.539094 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.539298 (+ 204us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.539360 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:04:18.539567 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.530395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34913 (local address 127.17.188.65:60689)
0620 23:04:18.530543 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.530579 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.530985 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.531207 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.531210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.531480 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:18.531744 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.531750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.532827 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.532828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.534221 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.534224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.534765 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.534767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.534912 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.535395 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:18.535403 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:18.537141 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:18.538470 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.538476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.538477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.538673 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.538823 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.538823 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.538823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.538911 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.539098 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:18.539102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:18.539204 (+ 102us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.539278 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.539388 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260620 23:04:18.539994 25917 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34913
I20260620 23:04:18.540074 25917 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:18.540251 25917 heartbeater.cc:511] Master 127.17.188.126:34913 requested a full tablet report, sending...
I20260620 23:04:18.540769 25713 ts_manager.cc:194] Registered new tserver with Master: da8933fda0cb410684faff151335307b (127.17.188.65:46681)
I20260620 23:04:18.541632 25713 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:60689
I20260620 23:04:18.542707 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 25786
I20260620 23:04:18.542810 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:04:18.542984 18161 external_mini_cluster.cc:1428] Setting key d97bdbb5e1b7fdc13c37e29d9e631216
I20260620 23:04:18.546623 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.541013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:38321 (local address 127.17.188.126:34913)
0620 23:04:18.541045 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.541046 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.541114 (+ 68us) server_negotiation.cc:407] Connection header received
0620 23:04:18.541176 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.541177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.541185 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.541238 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:18.541548 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.542148 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.542907 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.543175 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.543243 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.544100 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:18.544101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:18.544102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:18.544108 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:18.545566 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.545878 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.545879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.545880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.545933 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.546135 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.546136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.546137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.546242 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:18.546310 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.546466 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.546539 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:04:18.546640 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.540957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34913 (local address 127.17.188.65:38321)
0620 23:04:18.541062 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.541072 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.541124 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.541241 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.541242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.541438 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:18.541488 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.541490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.542239 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.542240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.542826 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.542829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.543261 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.543263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.543336 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.543733 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:18.543737 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:18.544011 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:18.545639 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.545639 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.545640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.545811 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.546003 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.546005 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.546005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.546065 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.546304 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:18.546305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:18.546348 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.546444 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.546539 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for oryx/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab.
Entry for principal oryx/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab.
I20260620 23:04:18.566071 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34913
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab
--principal=oryx/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:18.641469 25926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:18.641642 25926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:18.641664 25926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:18.642767 25926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:18.642824 25926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:18.642845 25926 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:04:18.642904 25926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:04:18.644320 25926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab
--principal=oryx/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.188.126:34913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:18.644541 25926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:18.644771 25926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:04:18.649297 25926 server_base.cc:1034] running on GCE node
W20260620 23:04:18.649379 25932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:18.649482 25931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:18.649477 25934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:18.649776 25926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:18.650043 25926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:18.651177 25926 hybrid_clock.cc:648] HybridClock initialized: now 1781996658651166 us; error 28 us; skew 500 ppm
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:18.653548 25926 init.cc:377] Logged in from keytab as oryx/127.17.188.66@KRBTEST.COM (short username oryx)
I20260620 23:04:18.654006 25926 webserver.cc:460] Webserver started at http://127.17.188.66:45251/ using document root <none> and password file <none>
I20260620 23:04:18.654245 25926 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:18.654286 25926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:18.654420 25926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:18.655536 25926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "f68c01abea2b46c1b6555f19496ec43c"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "4a9d3f2877509a670b969435594e3168"
server_key_iv: "73a826110bb5bd32028e478b3ff67812"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.655907 25926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "f68c01abea2b46c1b6555f19496ec43c"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "4a9d3f2877509a670b969435594e3168"
server_key_iv: "73a826110bb5bd32028e478b3ff67812"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.657177 25926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:04:18.657887 25941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.658048 25926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.658136 25926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "f68c01abea2b46c1b6555f19496ec43c"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "4a9d3f2877509a670b969435594e3168"
server_key_iv: "73a826110bb5bd32028e478b3ff67812"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.658211 25926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:18.674073 25926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:18.676198 25926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:18.676338 25926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:18.676796 25926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:18.676858 25926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.676909 25926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:18.676937 25926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:04:18.684024 25926 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:39731
I20260620 23:04:18.684311 26056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:39731 every 8 connection(s)
I20260620 23:04:18.684450 25926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:04:18.686370 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.677193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:47369 (local address 127.17.188.126:34913)
0620 23:04:18.677242 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.677244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.677475 (+ 231us) server_negotiation.cc:407] Connection header received
0620 23:04:18.678110 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.678111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.678122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.678197 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:18.679145 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.679772 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.681383 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.681722 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.681794 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.683749 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:18.683751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:18.683752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:18.683761 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:18.685283 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.685703 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.685703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.685704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.685752 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.685957 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.685957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.685957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.686045 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:18.686088 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.686258 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.686309 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:04:18.686496 25950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.677190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34913 (local address 127.17.188.66:47369)
0620 23:04:18.677345 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.677386 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.678013 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.678296 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.678300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.678736 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:18.679038 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.679044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.679842 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.679843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.681252 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.681255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.681821 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.681823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.681943 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.682398 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:18.682411 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:18.683658 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:18.685415 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.685421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.685423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.685642 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.685810 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.685811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.685811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.685895 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.686089 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:18.686092 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:18.686179 (+ 87us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.686248 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.686344 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
I20260620 23:04:18.686939 26057 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34913
I20260620 23:04:18.687009 26057 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:18.687168 26057 heartbeater.cc:511] Master 127.17.188.126:34913 requested a full tablet report, sending...
I20260620 23:04:18.687572 25714 ts_manager.cc:194] Registered new tserver with Master: f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731)
I20260620 23:04:18.688081 25714 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:47369
I20260620 23:04:18.689941 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 25926
I20260620 23:04:18.690043 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:04:18.690245 18161 external_mini_cluster.cc:1428] Setting key 60b715025d7ab04d21bcbe1f73641b42
I20260620 23:04:18.693651 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.687939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:49137 (local address 127.17.188.126:34913)
0620 23:04:18.687974 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.687976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.688075 (+ 99us) server_negotiation.cc:407] Connection header received
0620 23:04:18.688143 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.688143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.688152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.688201 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:18.688580 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.689080 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.689812 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.690095 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.690171 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.691103 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:18.691105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:18.691105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:18.691112 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:18.692568 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.692947 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.692947 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.692948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.692991 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.693167 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.693168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.693168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.693288 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:18.693397 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.693541 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.693595 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:04:18.693840 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.687889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34913 (local address 127.17.188.66:49137)
0620 23:04:18.688009 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.688018 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.688089 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.688269 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.688270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.688462 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:18.688511 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.688513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.689164 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.689165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.689736 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.689738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.690169 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.690170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.690234 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.690678 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:18.690683 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:18.691032 (+ 349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:18.692639 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.692640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.692640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.692871 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.693047 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.693047 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.693048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.693111 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.693393 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:18.693394 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:18.693444 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.693524 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.693627 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for oryx/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab.
Entry for principal oryx/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab.
I20260620 23:04:18.713039 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34913
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab
--principal=oryx/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:18.790872 26066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:18.791066 26066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:18.791088 26066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:18.792177 26066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:18.792215 26066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:18.792254 26066 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:04:18.792316 26066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:04:18.793773 26066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab
--principal=oryx/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.188.126:34913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:18.793995 26066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:18.794193 26066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:18.798753 26074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:18.798746 26071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:18.798733 26072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:18.798802 26066 server_base.cc:1034] running on GCE node
I20260620 23:04:18.799167 26066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:18.799361 26066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:18.800487 26066 hybrid_clock.cc:648] HybridClock initialized: now 1781996658800469 us; error 33 us; skew 500 ppm
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:18.802675 26066 init.cc:377] Logged in from keytab as oryx/127.17.188.67@KRBTEST.COM (short username oryx)
I20260620 23:04:18.803107 26066 webserver.cc:460] Webserver started at http://127.17.188.67:37843/ using document root <none> and password file <none>
I20260620 23:04:18.803310 26066 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:18.803354 26066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:18.803490 26066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:18.804448 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "927abf2707534570b8ee54b52872516c"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "04c31575875c8d6faa375e8bb3c06e28"
server_key_iv: "8f68e0d2090ce8b0842923bd61dbe57a"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.804807 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "927abf2707534570b8ee54b52872516c"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "04c31575875c8d6faa375e8bb3c06e28"
server_key_iv: "8f68e0d2090ce8b0842923bd61dbe57a"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.806129 26066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:04:18.806782 26081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.806948 26066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:04:18.807029 26066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "927abf2707534570b8ee54b52872516c"
format_stamp: "Formatted at 2026-06-20 23:04:18 on dist-test-slave-zpss"
server_key: "04c31575875c8d6faa375e8bb3c06e28"
server_key_iv: "8f68e0d2090ce8b0842923bd61dbe57a"
server_key_version: "encryptionkey@0"
I20260620 23:04:18.807092 26066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:18.830013 26066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:18.831610 26066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:18.831768 26066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:18.832259 26066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:18.832296 26066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:18.832317 26066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:18.832329 26066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:04:18.839474 26066 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:39015
I20260620 23:04:18.839769 26066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:04:18.840662 26196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:39015 every 8 connection(s)
I20260620 23:04:18.841681 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.832597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:58145 (local address 127.17.188.126:34913)
0620 23:04:18.832658 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.832661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.832880 (+ 219us) server_negotiation.cc:407] Connection header received
0620 23:04:18.833312 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.833313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.833393 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.833466 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:18.834409 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.834895 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.835857 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.836051 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.836122 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.838610 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:18.838612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:18.838612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:18.838620 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:18.840068 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.840591 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.840592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.840593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.840681 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.840966 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.840967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.840968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.841093 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:18.841167 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.841496 (+ 329us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.841593 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:04:18.841813 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.832604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34913 (local address 127.17.188.67:58145)
0620 23:04:18.832750 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.832788 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.833224 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.833609 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.833613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.833994 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:18.834290 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.834297 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.834985 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.834986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.835746 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.835748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.836127 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.836128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.836272 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.836772 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:18.836785 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:18.838523 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:18.840213 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.840215 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.840217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.840526 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.840767 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.840768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.840768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.840888 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.841176 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:18.841182 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:18.841306 (+ 124us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.841458 (+ 152us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.841613 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260620 23:04:18.842227 26197 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34913
I20260620 23:04:18.842293 26197 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:18.842463 26197 heartbeater.cc:511] Master 127.17.188.126:34913 requested a full tablet report, sending...
I20260620 23:04:18.842895 25713 ts_manager.cc:194] Registered new tserver with Master: 927abf2707534570b8ee54b52872516c (127.17.188.67:39015)
I20260620 23:04:18.843369 25713 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:58145
I20260620 23:04:18.846928 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 26066
I20260620 23:04:18.847038 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:04:18.847249 18161 external_mini_cluster.cc:1428] Setting key 2ee93f5fad76a745801d74a199ea4402
I20260620 23:04:18.848104 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:04:18.848502 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.843225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:36973 (local address 127.17.188.126:34913)
0620 23:04:18.843260 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.843261 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.843382 (+ 121us) server_negotiation.cc:407] Connection header received
0620 23:04:18.843474 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.843475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.843484 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.843540 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:18.843837 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.844281 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.844952 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.845142 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.845215 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.846082 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:18.846083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:18.846083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:18.846089 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:18.847513 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.847878 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.847878 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.847879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.847936 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.848116 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.848117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.848117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.848218 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:18.848279 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.848410 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.848457 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:04:18.848570 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.843177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34913 (local address 127.17.188.67:36973)
0620 23:04:18.843322 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.843331 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.843396 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.843544 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.843545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.843710 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:18.843766 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.843767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.844375 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.844376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.844866 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.844867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.845216 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.845217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.845289 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.845718 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:18.845723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:18.845988 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:18.847598 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.847599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.847599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.847806 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.847994 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:18.847995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:18.847996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:18.848057 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:18.848289 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:18.848290 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:18.848329 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.848396 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.848474 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 23:04:18.867616 25763 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:18.862986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.17.188.126:34913)
0620 23:04:18.863047 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.863049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.863058 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:18.863095 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.863095 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.863108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.863147 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:18.863611 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.864043 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.864718 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.864895 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.864940 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.867549 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45812: BlockingRecv error: recv got EOF from 127.0.0.1:45812 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 20 23:04:18 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:04:18.874137 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.868272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.17.188.126:34913)
0620 23:04:18.868316 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.868318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.868375 (+ 57us) server_negotiation.cc:407] Connection header received
0620 23:04:18.868411 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.868412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.868418 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.868455 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:18.868914 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.869379 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.870075 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.870237 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.870286 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.872060 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:18.872061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:18.872062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:18.872067 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:18.873307 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.873618 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.873619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.873619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.873665 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:18.873810 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:18.873810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:18.873810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:18.873900 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:18.873963 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.874038 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.874087 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:04:18.875452 25713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:04:18.876821 25713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:04:18.882391 26216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.878746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:39731 (local address 127.0.0.1:40480)
0620 23:04:18.878908 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.878944 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.879000 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.879591 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.879592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.879594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:18.879632 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.879633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.880666 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.880667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.881437 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.881439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.882121 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.882122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.882196 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.882208 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.882277 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.882323 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260620 23:04:18.882421 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.878844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40480 (local address 127.17.188.66:39731)
0620 23:04:18.879389 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.879393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.879402 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:18.879437 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.879439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.879453 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.879532 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:18.879700 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.880600 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.881536 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.882041 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.882121 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.882191 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.882299 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.882333 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":71,"threads_started":1}
I20260620 23:04:18.882735 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.878939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:46681 (local address 127.0.0.1:43290)
0620 23:04:18.879095 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.879104 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.879141 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.879412 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.879413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.879415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:18.879464 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.879466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.880293 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.880294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.881525 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.881528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.882472 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.882473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.882553 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.882558 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.882607 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.882648 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":4,"thread_start_us":47,"threads_started":1}
I20260620 23:04:18.882939 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.878828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56044 (local address 127.17.188.67:39015)
0620 23:04:18.879143 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.879149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.879158 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:18.879191 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.879193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.879205 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.879278 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:18.879445 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.880404 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.881621 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.882299 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.882374 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.882436 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.882638 (+ 202us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.882676 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":83,"threads_started":1}
I20260620 23:04:18.883054 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.879132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.17.188.65:46681)
0620 23:04:18.879267 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.879271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.879280 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:18.879317 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.879319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.879330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.879414 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:18.879552 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.880216 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.881623 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.882295 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.882365 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.882436 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.882894 (+ 458us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.882930 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260620 23:04:18.882736 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.878746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:39015 (local address 127.0.0.1:56044)
0620 23:04:18.879020 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.879029 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.879092 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.879302 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.879305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.879307 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:18.879362 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.879364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.880476 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.880476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.881525 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.881528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.882474 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.882475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.882553 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.882557 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.882613 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.882647 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":32,"spinlock_wait_cycles":1408,"thread_start_us":66,"threads_started":1}
I20260620 23:04:18.883350 25991 tablet_service.cc:1467] Processing CreateTablet for tablet fb98cf1c2b094aaaa6f82a384f300396 (DEFAULT_TABLE table=test-table [id=cf7411c15c6d4fd5b6c5520c31a39163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:18.883708 25991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb98cf1c2b094aaaa6f82a384f300396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:18.883776 25851 tablet_service.cc:1467] Processing CreateTablet for tablet fb98cf1c2b094aaaa6f82a384f300396 (DEFAULT_TABLE table=test-table [id=cf7411c15c6d4fd5b6c5520c31a39163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:18.884105 25851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb98cf1c2b094aaaa6f82a384f300396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:18.885699 26131 tablet_service.cc:1467] Processing CreateTablet for tablet fb98cf1c2b094aaaa6f82a384f300396 (DEFAULT_TABLE table=test-table [id=cf7411c15c6d4fd5b6c5520c31a39163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:18.886027 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb98cf1c2b094aaaa6f82a384f300396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:18.886947 26222 tablet_bootstrap.cc:492] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: Bootstrap starting.
I20260620 23:04:18.887373 26223 tablet_bootstrap.cc:492] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: Bootstrap starting.
I20260620 23:04:18.887581 26222 tablet_bootstrap.cc:654] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:18.887921 26222 log.cc:824] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:18.888022 26223 tablet_bootstrap.cc:654] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:18.888321 26223 log.cc:824] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:18.888772 26222 tablet_bootstrap.cc:492] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: No bootstrap required, opened a new log
I20260620 23:04:18.888844 26222 ts_tablet_manager.cc:1397] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:04:18.889021 26223 tablet_bootstrap.cc:492] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: No bootstrap required, opened a new log
I20260620 23:04:18.889081 26223 ts_tablet_manager.cc:1397] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:04:18.889619 26226 tablet_bootstrap.cc:492] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: Bootstrap starting.
I20260620 23:04:18.890196 26226 tablet_bootstrap.cc:654] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:18.890465 26226 log.cc:824] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:18.891068 26226 tablet_bootstrap.cc:492] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: No bootstrap required, opened a new log
I20260620 23:04:18.891022 26222 raft_consensus.cc:348] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } }
I20260620 23:04:18.891005 26223 raft_consensus.cc:348] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } }
I20260620 23:04:18.891134 26226 ts_tablet_manager.cc:1397] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:04:18.891168 26223 raft_consensus.cc:374] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:18.891168 26222 raft_consensus.cc:374] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:18.891227 26222 raft_consensus.cc:729] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f68c01abea2b46c1b6555f19496ec43c, State: Initialized, Role: FOLLOWER
I20260620 23:04:18.891251 26223 raft_consensus.cc:729] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8933fda0cb410684faff151335307b, State: Initialized, Role: FOLLOWER
I20260620 23:04:18.891366 26223 consensus_queue.cc:260] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } }
I20260620 23:04:18.891381 26222 consensus_queue.cc:260] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } }
I20260620 23:04:18.891618 26222 ts_tablet_manager.cc:1428] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:04:18.891618 26223 ts_tablet_manager.cc:1428] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:04:18.891721 25917 heartbeater.cc:503] Master 127.17.188.126:34913 was elected leader, sending a full tablet report...
I20260620 23:04:18.891721 26057 heartbeater.cc:503] Master 127.17.188.126:34913 was elected leader, sending a full tablet report...
I20260620 23:04:18.892973 26226 raft_consensus.cc:348] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } }
I20260620 23:04:18.893121 26226 raft_consensus.cc:374] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:18.893163 26226 raft_consensus.cc:729] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 927abf2707534570b8ee54b52872516c, State: Initialized, Role: FOLLOWER
I20260620 23:04:18.893270 26226 consensus_queue.cc:260] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } }
I20260620 23:04:18.893513 26226 ts_tablet_manager.cc:1428] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:04:18.893565 26197 heartbeater.cc:503] Master 127.17.188.126:34913 was elected leader, sending a full tablet report...
W20260620 23:04:18.935724 26058 tablet.cc:2369] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:18.970815 26228 raft_consensus.cc:482] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:04:18.970958 26228 raft_consensus.cc:504] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } }
I20260620 23:04:18.971326 26228 leader_election.cc:288] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 927abf2707534570b8ee54b52872516c (127.17.188.67:39015), f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731)
I20260620 23:04:18.973964 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.971538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:52569 (local address 127.17.188.67:39015)
0620 23:04:18.971577 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.971579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.971588 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:18.971615 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.971616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.971624 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.971693 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:18.971833 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.972301 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.973199 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.973612 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.973686 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.973695 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.973860 (+ 165us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.973899 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:04:18.973979 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.971452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:39015 (local address 127.17.188.65:52569)
0620 23:04:18.971505 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.971517 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.971561 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.971696 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.971697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.971698 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:18.971767 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.971769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.972390 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.972391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.973109 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.973111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.973696 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.973698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.973785 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.973788 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.973848 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.973897 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1}
I20260620 23:04:18.974063 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.971570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:48113 (local address 127.17.188.66:39731)
0620 23:04:18.971610 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.971612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.971723 (+ 111us) server_negotiation.cc:407] Connection header received
0620 23:04:18.971800 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.971801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.971810 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.971877 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:18.972026 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.972471 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.973367 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.973717 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.973792 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.973801 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.973966 (+ 165us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.973999 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:04:18.974079 26231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.971482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:39731 (local address 127.17.188.65:48113)
0620 23:04:18.971651 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:18.971665 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:18.971732 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:18.971879 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:18.971880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:18.971882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:18.971960 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.971961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.972549 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.972550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.973256 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:18.973258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.973811 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.973812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:18.973887 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.973891 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:04:18.973946 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 23:04:18.973985 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":110,"threads_started":1}
I20260620 23:04:18.974309 26151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb98cf1c2b094aaaa6f82a384f300396" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "927abf2707534570b8ee54b52872516c" is_pre_election: true
I20260620 23:04:18.974342 26011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb98cf1c2b094aaaa6f82a384f300396" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68c01abea2b46c1b6555f19496ec43c" is_pre_election: true
I20260620 23:04:18.974450 26151 raft_consensus.cc:2454] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 0.
I20260620 23:04:18.974472 26011 raft_consensus.cc:2454] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 0.
I20260620 23:04:18.974639 25805 leader_election.cc:302] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 927abf2707534570b8ee54b52872516c, da8933fda0cb410684faff151335307b; no voters:
I20260620 23:04:18.974758 26228 raft_consensus.cc:2781] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:04:18.974810 26228 raft_consensus.cc:482] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:04:18.974838 26228 raft_consensus.cc:3037] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:18.975742 26228 raft_consensus.cc:504] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } }
I20260620 23:04:18.975872 26228 leader_election.cc:288] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 election: Requested vote from peers 927abf2707534570b8ee54b52872516c (127.17.188.67:39015), f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731)
I20260620 23:04:18.976028 26151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb98cf1c2b094aaaa6f82a384f300396" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "927abf2707534570b8ee54b52872516c"
I20260620 23:04:18.976032 26011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb98cf1c2b094aaaa6f82a384f300396" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68c01abea2b46c1b6555f19496ec43c"
I20260620 23:04:18.976102 26011 raft_consensus.cc:3037] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:18.976102 26151 raft_consensus.cc:3037] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:18.976951 26011 raft_consensus.cc:2454] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 1.
I20260620 23:04:18.976951 26151 raft_consensus.cc:2454] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 1.
I20260620 23:04:18.977133 25803 leader_election.cc:302] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da8933fda0cb410684faff151335307b, f68c01abea2b46c1b6555f19496ec43c; no voters:
I20260620 23:04:18.977229 26228 raft_consensus.cc:2781] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:18.977391 26228 raft_consensus.cc:686] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 1 LEADER]: Becoming Leader. State: Replica: da8933fda0cb410684faff151335307b, State: Running, Role: LEADER
I20260620 23:04:18.977485 26228 consensus_queue.cc:237] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } }
I20260620 23:04:18.978268 25714 catalog_manager.cc:5495] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b reported cstate change: term changed from 0 to 1, leader changed from <none> to da8933fda0cb410684faff151335307b (127.17.188.65). New cstate: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } health_report { overall_health: HEALTHY } } }
I20260620 23:04:18.996417 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:18.994137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.17.188.65:46681)
0620 23:04:18.994190 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:18.994192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:18.994201 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:18.994269 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:18.994270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:18.994288 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:18.994355 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:18.994505 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.994953 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.995713 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:18.995917 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:18.995985 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:18.996137 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:18.996230 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:18.996312 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:18.996333 (+ 21us) server_negotiation.cc:299] Negotiation successful
0620 23:04:18.996351 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:04:18.997864 26151 raft_consensus.cc:1261] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Refusing update from remote peer da8933fda0cb410684faff151335307b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:18.997895 26011 raft_consensus.cc:1261] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Refusing update from remote peer da8933fda0cb410684faff151335307b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:18.998190 26228 consensus_queue.cc:1035] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [LEADER]: Connected to new peer: Peer: permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:18.998315 26232 consensus_queue.cc:1035] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:19.000828 26236 mvcc.cc:205] Tried to move back new op lower bound from 7299058315253182464 to 7299058315172290560. Current Snapshot: MvccSnapshot[applied={T|T < 7299058315253182464}]
I20260620 23:04:19.001000 26237 mvcc.cc:205] Tried to move back new op lower bound from 7299058315253182464 to 7299058315172290560. Current Snapshot: MvccSnapshot[applied={T|T < 7299058315253182464 or (T in {7299058315253182464})}]
I20260620 23:04:19.002388 25714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:19.002466 25714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:19.003121 25714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943: Sending DeleteTablet for 3 replicas of tablet fb98cf1c2b094aaaa6f82a384f300396
I20260620 23:04:19.003437 26131 tablet_service.cc:1514] Processing DeleteTablet for tablet fb98cf1c2b094aaaa6f82a384f300396 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:18 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56044
I20260620 23:04:19.003465 25851 tablet_service.cc:1514] Processing DeleteTablet for tablet fb98cf1c2b094aaaa6f82a384f300396 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:18 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43290
I20260620 23:04:19.003585 25991 tablet_service.cc:1514] Processing DeleteTablet for tablet fb98cf1c2b094aaaa6f82a384f300396 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:18 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:40480
I20260620 23:04:19.003707 26245 tablet_replica.cc:330] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: stopping tablet replica
I20260620 23:04:19.003789 26245 raft_consensus.cc:2229] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:19.003846 26247 tablet_replica.cc:330] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: stopping tablet replica
I20260620 23:04:19.003937 26246 tablet_replica.cc:330] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: stopping tablet replica
I20260620 23:04:19.003894 25714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 23:04:19.003993 26247 raft_consensus.cc:2229] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260620 23:04:19.004009 25714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:04:19.004098 26247 raft_consensus.cc:2258] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.004227 26246 raft_consensus.cc:2229] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:04:19.004546 26247 ts_tablet_manager.cc:1905] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.004698 26246 raft_consensus.cc:2258] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.005159 26246 ts_tablet_manager.cc:1905] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.005224 26131 tablet_service.cc:1467] Processing CreateTablet for tablet 85881397249146ea99262b41748935c3 (DEFAULT_TABLE table=test-table [id=7cd2e0e7bf5b4e33a086254627e3b5af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:19.005363 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85881397249146ea99262b41748935c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.005694 25991 tablet_service.cc:1467] Processing CreateTablet for tablet 85881397249146ea99262b41748935c3 (DEFAULT_TABLE table=test-table [id=7cd2e0e7bf5b4e33a086254627e3b5af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:19.005793 25991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85881397249146ea99262b41748935c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.006317 26247 ts_tablet_manager.cc:1918] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:19.006372 26247 log.cc:1196] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/fb98cf1c2b094aaaa6f82a384f300396
I20260620 23:04:19.006597 26247 ts_tablet_manager.cc:1939] T fb98cf1c2b094aaaa6f82a384f300396 P f68c01abea2b46c1b6555f19496ec43c: Deleting consensus metadata
I20260620 23:04:19.006644 26226 tablet_bootstrap.cc:492] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: Bootstrap starting.
I20260620 23:04:19.004613 26245 raft_consensus.cc:2258] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.006891 26222 tablet_bootstrap.cc:492] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: Bootstrap starting.
I20260620 23:04:19.006968 25699 catalog_manager.cc:4841] TS f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731): tablet fb98cf1c2b094aaaa6f82a384f300396 (table test-table [id=cf7411c15c6d4fd5b6c5520c31a39163]) successfully deleted
I20260620 23:04:19.007063 26226 tablet_bootstrap.cc:654] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.007247 26222 tablet_bootstrap.cc:654] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.007615 25851 tablet_service.cc:1467] Processing CreateTablet for tablet 85881397249146ea99262b41748935c3 (DEFAULT_TABLE table=test-table [id=7cd2e0e7bf5b4e33a086254627e3b5af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:19.007665 26226 tablet_bootstrap.cc:492] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: No bootstrap required, opened a new log
I20260620 23:04:19.007714 26226 ts_tablet_manager.cc:1397] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:19.007730 25851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85881397249146ea99262b41748935c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.007844 26226 raft_consensus.cc:348] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.007884 26226 raft_consensus.cc:374] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.007896 26226 raft_consensus.cc:729] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 927abf2707534570b8ee54b52872516c, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.007947 26226 consensus_queue.cc:260] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.008036 26226 ts_tablet_manager.cc:1428] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:19.008419 26246 ts_tablet_manager.cc:1918] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:19.008476 26246 log.cc:1196] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/fb98cf1c2b094aaaa6f82a384f300396
I20260620 23:04:19.008703 26246 ts_tablet_manager.cc:1939] T fb98cf1c2b094aaaa6f82a384f300396 P da8933fda0cb410684faff151335307b: Deleting consensus metadata
I20260620 23:04:19.009063 26223 tablet_bootstrap.cc:492] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: Bootstrap starting.
I20260620 23:04:19.009482 26223 tablet_bootstrap.cc:654] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.009740 26222 tablet_bootstrap.cc:492] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: No bootstrap required, opened a new log
I20260620 23:04:19.009786 26222 ts_tablet_manager.cc:1397] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 23:04:19.009817 25699 catalog_manager.cc:4841] TS da8933fda0cb410684faff151335307b (127.17.188.65:46681): tablet fb98cf1c2b094aaaa6f82a384f300396 (table test-table [id=cf7411c15c6d4fd5b6c5520c31a39163]) successfully deleted
I20260620 23:04:19.009917 26222 raft_consensus.cc:348] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.009999 26222 raft_consensus.cc:374] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.010018 26222 raft_consensus.cc:729] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f68c01abea2b46c1b6555f19496ec43c, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.010074 26222 consensus_queue.cc:260] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.010108 26245 ts_tablet_manager.cc:1905] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.010186 26222 ts_tablet_manager.cc:1428] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:19.011418 26245 ts_tablet_manager.cc:1918] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:19.011485 26245 log.cc:1196] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/fb98cf1c2b094aaaa6f82a384f300396
I20260620 23:04:19.011737 26245 ts_tablet_manager.cc:1939] T fb98cf1c2b094aaaa6f82a384f300396 P 927abf2707534570b8ee54b52872516c: Deleting consensus metadata
I20260620 23:04:19.012128 25701 catalog_manager.cc:4841] TS 927abf2707534570b8ee54b52872516c (127.17.188.67:39015): tablet fb98cf1c2b094aaaa6f82a384f300396 (table test-table [id=cf7411c15c6d4fd5b6c5520c31a39163]) successfully deleted
I20260620 23:04:19.012218 26223 tablet_bootstrap.cc:492] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: No bootstrap required, opened a new log
I20260620 23:04:19.012262 26223 ts_tablet_manager.cc:1397] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260620 23:04:19.012395 26223 raft_consensus.cc:348] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.012459 26223 raft_consensus.cc:374] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.012491 26223 raft_consensus.cc:729] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8933fda0cb410684faff151335307b, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.012537 26223 consensus_queue.cc:260] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.012622 26223 ts_tablet_manager.cc:1428] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260620 23:04:19.037420 25918 tablet.cc:2369] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:19.074008 26240 raft_consensus.cc:482] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:04:19.074100 26240 raft_consensus.cc:504] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.074257 26240 leader_election.cc:288] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731), 927abf2707534570b8ee54b52872516c (127.17.188.67:39015)
I20260620 23:04:19.074430 26011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85881397249146ea99262b41748935c3" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68c01abea2b46c1b6555f19496ec43c" is_pre_election: true
I20260620 23:04:19.074448 26151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85881397249146ea99262b41748935c3" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "927abf2707534570b8ee54b52872516c" is_pre_election: true
I20260620 23:04:19.074532 26011 raft_consensus.cc:2454] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 0.
I20260620 23:04:19.074548 26151 raft_consensus.cc:2454] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 0.
I20260620 23:04:19.074707 25805 leader_election.cc:302] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 927abf2707534570b8ee54b52872516c, da8933fda0cb410684faff151335307b; no voters:
I20260620 23:04:19.074800 26240 raft_consensus.cc:2781] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:04:19.074856 26240 raft_consensus.cc:482] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:04:19.074887 26240 raft_consensus.cc:3037] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.075519 26240 raft_consensus.cc:504] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.075654 26240 leader_election.cc:288] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 election: Requested vote from peers f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731), 927abf2707534570b8ee54b52872516c (127.17.188.67:39015)
I20260620 23:04:19.075778 26011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85881397249146ea99262b41748935c3" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68c01abea2b46c1b6555f19496ec43c"
I20260620 23:04:19.075848 26011 raft_consensus.cc:3037] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.075887 26151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85881397249146ea99262b41748935c3" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "927abf2707534570b8ee54b52872516c"
I20260620 23:04:19.075956 26151 raft_consensus.cc:3037] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.076362 26011 raft_consensus.cc:2454] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 1.
I20260620 23:04:19.076521 25803 leader_election.cc:302] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da8933fda0cb410684faff151335307b, f68c01abea2b46c1b6555f19496ec43c; no voters:
I20260620 23:04:19.076618 26240 raft_consensus.cc:2781] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:19.076661 26240 raft_consensus.cc:686] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 1 LEADER]: Becoming Leader. State: Replica: da8933fda0cb410684faff151335307b, State: Running, Role: LEADER
I20260620 23:04:19.076709 26151 raft_consensus.cc:2454] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 1.
I20260620 23:04:19.076713 26240 consensus_queue.cc:237] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.077258 25714 catalog_manager.cc:5495] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b reported cstate change: term changed from 0 to 1, leader changed from <none> to da8933fda0cb410684faff151335307b (127.17.188.65). New cstate: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } health_report { overall_health: UNKNOWN } } }
W20260620 23:04:19.091080 26198 tablet.cc:2369] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 23:04:19 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.126@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:04:19.105149 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.098122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45820 (local address 127.17.188.126:34913)
0620 23:04:19.098175 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.098177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.098294 (+ 117us) server_negotiation.cc:407] Connection header received
0620 23:04:19.098369 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.098369 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.098378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.098441 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:19.098871 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.099360 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.100013 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.100187 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.100263 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.102598 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:19.102600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:19.102600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:19.102606 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:19.104002 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:19.104411 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:19.104411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:19.104412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:19.104459 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:19.104641 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:19.104642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:19.104642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:19.104735 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:19.104783 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.104936 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.105064 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":28}
I20260620 23:04:19.105154 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.098054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34913 (local address 127.0.0.1:45820)
0620 23:04:19.098227 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:19.098236 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:19.098312 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:19.098447 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:19.098447 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:19.098734 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:19.098787 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.098789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.099431 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.099432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.099930 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.099932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.100263 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.100264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.100338 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.100860 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:19.100865 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:19.102507 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:19.104080 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:19.104086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:19.104088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:19.104332 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:19.104517 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:19.104518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:19.104518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:19.104585 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:19.104789 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:19.104794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:19.104844 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 23:04:19.104912 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:04:19.105021 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260620 23:04:19.107244 25714 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:45820:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 23:04:19.107460 25714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:04:19.108682 25851 tablet_service.cc:1467] Processing CreateTablet for tablet 4f2d1627c20e493fb5153d2c54803688 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b913c71f40ef486580ca486f7642dd5f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:04:19.108727 26131 tablet_service.cc:1467] Processing CreateTablet for tablet 4f2d1627c20e493fb5153d2c54803688 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b913c71f40ef486580ca486f7642dd5f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:04:19.108727 25991 tablet_service.cc:1467] Processing CreateTablet for tablet 4f2d1627c20e493fb5153d2c54803688 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b913c71f40ef486580ca486f7642dd5f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:04:19.108824 25851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f2d1627c20e493fb5153d2c54803688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.108860 25991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f2d1627c20e493fb5153d2c54803688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.108860 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f2d1627c20e493fb5153d2c54803688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.110347 26226 tablet_bootstrap.cc:492] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: Bootstrap starting.
I20260620 23:04:19.110742 26226 tablet_bootstrap.cc:654] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.111057 26222 tablet_bootstrap.cc:492] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: Bootstrap starting.
I20260620 23:04:19.111361 26226 tablet_bootstrap.cc:492] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: No bootstrap required, opened a new log
I20260620 23:04:19.111421 26226 ts_tablet_manager.cc:1397] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:19.111429 26222 tablet_bootstrap.cc:654] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.111577 26226 raft_consensus.cc:348] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.111644 26226 raft_consensus.cc:374] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.111663 26226 raft_consensus.cc:729] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 927abf2707534570b8ee54b52872516c, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.111703 26226 consensus_queue.cc:260] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.111716 26223 tablet_bootstrap.cc:492] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: Bootstrap starting.
I20260620 23:04:19.111781 26226 ts_tablet_manager.cc:1428] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:19.111902 26230 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.111985 26230 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:04:19.112002 26223 tablet_bootstrap.cc:654] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.112572 26223 tablet_bootstrap.cc:492] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: No bootstrap required, opened a new log
I20260620 23:04:19.112630 26223 ts_tablet_manager.cc:1397] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 23:04:19.112737 26223 raft_consensus.cc:348] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.112799 26223 raft_consensus.cc:374] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.112812 26223 raft_consensus.cc:729] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8933fda0cb410684faff151335307b, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.112860 26223 consensus_queue.cc:260] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.112943 26223 ts_tablet_manager.cc:1428] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:19.112998 26228 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.113121 26228 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:04:19.113598 26222 tablet_bootstrap.cc:492] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: No bootstrap required, opened a new log
I20260620 23:04:19.113660 26222 ts_tablet_manager.cc:1397] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 23:04:19.113817 26222 raft_consensus.cc:348] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.113888 26222 raft_consensus.cc:374] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.113901 26222 raft_consensus.cc:729] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f68c01abea2b46c1b6555f19496ec43c, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.113951 26222 consensus_queue.cc:260] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.114039 26222 ts_tablet_manager.cc:1428] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:19.114099 26229 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.114187 26229 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:04:19.307585 26228 raft_consensus.cc:482] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:04:19.307673 26228 raft_consensus.cc:504] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.307822 26228 leader_election.cc:288] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731), 927abf2707534570b8ee54b52872516c (127.17.188.67:39015)
I20260620 23:04:19.307993 26011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f2d1627c20e493fb5153d2c54803688" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68c01abea2b46c1b6555f19496ec43c" is_pre_election: true
I20260620 23:04:19.308075 26011 raft_consensus.cc:2454] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 0.
I20260620 23:04:19.308035 26151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f2d1627c20e493fb5153d2c54803688" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "927abf2707534570b8ee54b52872516c" is_pre_election: true
I20260620 23:04:19.308152 26151 raft_consensus.cc:2454] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 0.
I20260620 23:04:19.308228 25803 leader_election.cc:302] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da8933fda0cb410684faff151335307b, f68c01abea2b46c1b6555f19496ec43c; no voters:
I20260620 23:04:19.308318 26228 raft_consensus.cc:2781] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:04:19.308363 26228 raft_consensus.cc:482] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:04:19.308393 26228 raft_consensus.cc:3037] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.309051 26228 raft_consensus.cc:504] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.309180 26228 leader_election.cc:288] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 election: Requested vote from peers f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731), 927abf2707534570b8ee54b52872516c (127.17.188.67:39015)
I20260620 23:04:19.309336 26151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f2d1627c20e493fb5153d2c54803688" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "927abf2707534570b8ee54b52872516c"
I20260620 23:04:19.309355 26011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f2d1627c20e493fb5153d2c54803688" candidate_uuid: "da8933fda0cb410684faff151335307b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68c01abea2b46c1b6555f19496ec43c"
I20260620 23:04:19.309410 26011 raft_consensus.cc:3037] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.309423 26151 raft_consensus.cc:3037] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.309975 26011 raft_consensus.cc:2454] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 1.
I20260620 23:04:19.310058 26151 raft_consensus.cc:2454] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da8933fda0cb410684faff151335307b in term 1.
I20260620 23:04:19.310130 25803 leader_election.cc:302] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da8933fda0cb410684faff151335307b, f68c01abea2b46c1b6555f19496ec43c; no voters:
I20260620 23:04:19.310215 26228 raft_consensus.cc:2781] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:19.310278 26228 raft_consensus.cc:686] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [term 1 LEADER]: Becoming Leader. State: Replica: da8933fda0cb410684faff151335307b, State: Running, Role: LEADER
I20260620 23:04:19.310328 26228 consensus_queue.cc:237] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.310591 26240 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: TxnStatusTablet state changed. Reason: New leader da8933fda0cb410684faff151335307b. Latest consensus state: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.310703 26240 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:04:19.310886 26262 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 23:04:19.310904 25714 catalog_manager.cc:5495] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b reported cstate change: term changed from 0 to 1, leader changed from <none> to da8933fda0cb410684faff151335307b (127.17.188.65). New cstate: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } health_report { overall_health: UNKNOWN } } }
I20260620 23:04:19.311007 26262 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 23:04:19.343997 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.341197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:46681 (local address 127.0.0.1:43310)
0620 23:04:19.341275 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:19.341286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:19.341370 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:19.341558 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:19.341560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:19.341561 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:04:19.341603 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.341605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.342555 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.342556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.343192 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.343194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.343641 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.343642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.343688 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.343701 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.343867 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.343873 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:04:19.343904 (+ 31us) client_negotiation.cc:240] Negotiation successful
0620 23:04:19.343940 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 23:04:19.344012 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.341238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.17.188.65:46681)
0620 23:04:19.341278 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.341280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.341358 (+ 78us) server_negotiation.cc:407] Connection header received
0620 23:04:19.341462 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.341463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.341473 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.341521 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:19.341721 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.342484 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.343265 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.343571 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.343658 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.343772 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.343804 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.343856 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.343917 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.343939 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:04:19.344918 26011 raft_consensus.cc:1261] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Refusing update from remote peer da8933fda0cb410684faff151335307b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:19.344925 26151 raft_consensus.cc:1261] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Refusing update from remote peer da8933fda0cb410684faff151335307b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:19.345090 26228 consensus_queue.cc:1035] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:19.345242 26228 consensus_queue.cc:1035] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b [LEADER]: Connected to new peer: Peer: permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:19.345573 26229 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: TxnStatusTablet state changed. Reason: New leader da8933fda0cb410684faff151335307b. Latest consensus state: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.345664 26229 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:04:19.346051 26230 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: TxnStatusTablet state changed. Reason: New leader da8933fda0cb410684faff151335307b. Latest consensus state: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.346130 26230 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:04:19.346611 26228 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.346688 26228 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:04:19.346917 26235 mvcc.cc:205] Tried to move back new op lower bound from 7299058316675198976 to 7299058316535488512. Current Snapshot: MvccSnapshot[applied={T|T < 7299058316675198976 or (T in {7299058316675198976})}]
I20260620 23:04:19.347388 26230 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.347467 26230 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P 927abf2707534570b8ee54b52872516c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:04:19.348124 26232 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.348196 26232 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P da8933fda0cb410684faff151335307b: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:04:19.348332 26229 tablet_replica.cc:439] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da8933fda0cb410684faff151335307b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } } }
I20260620 23:04:19.348402 26229 tablet_replica.cc:442] T 4f2d1627c20e493fb5153d2c54803688 P f68c01abea2b46c1b6555f19496ec43c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:04:19.352468 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.350445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:51201 (local address 127.17.188.65:46681)
0620 23:04:19.350492 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.350494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.350522 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:04:19.350589 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.350589 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.350598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.350648 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:19.350744 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.351153 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.351838 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.352020 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.352094 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.352238 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.352268 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.352323 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.352384 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.352406 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":29}
I20260620 23:04:19.352521 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.350286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:46681 (local address 127.17.188.65:51201)
0620 23:04:19.350455 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:19.350478 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:19.350537 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:19.350651 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:19.350652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:19.350653 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:04:19.350690 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.350692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.351225 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.351226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.351754 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.351755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.352094 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.352095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.352141 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.352153 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.352322 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.352324 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:04:19.352381 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:04:19.352425 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260620 23:04:19.354346 26151 raft_consensus.cc:1261] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Refusing update from remote peer da8933fda0cb410684faff151335307b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:19.354353 26011 raft_consensus.cc:1261] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Refusing update from remote peer da8933fda0cb410684faff151335307b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:19.354560 26240 consensus_queue.cc:1035] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:19.354626 26228 consensus_queue.cc:1035] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [LEADER]: Connected to new peer: Peer: permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:19.364497 25713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:19.364585 25713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:19.365126 25713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943: Sending DeleteTablet for 3 replicas of tablet 85881397249146ea99262b41748935c3
I20260620 23:04:19.365366 26131 tablet_service.cc:1514] Processing DeleteTablet for tablet 85881397249146ea99262b41748935c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:19 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56044
I20260620 23:04:19.365409 25991 tablet_service.cc:1514] Processing DeleteTablet for tablet 85881397249146ea99262b41748935c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:19 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:40480
I20260620 23:04:19.365455 26245 tablet_replica.cc:330] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: stopping tablet replica
I20260620 23:04:19.365489 26247 tablet_replica.cc:330] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: stopping tablet replica
I20260620 23:04:19.365504 26245 raft_consensus.cc:2229] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:19.365525 26247 raft_consensus.cc:2229] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:19.365535 26245 raft_consensus.cc:2258] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.365554 26247 raft_consensus.cc:2258] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.365823 26245 ts_tablet_manager.cc:1905] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.368126 25850 tablet_service.cc:1514] Processing DeleteTablet for tablet 85881397249146ea99262b41748935c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:19 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43290
I20260620 23:04:19.368229 26246 tablet_replica.cc:330] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: stopping tablet replica
I20260620 23:04:19.368294 26246 raft_consensus.cc:2229] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:04:19.368331 26245 ts_tablet_manager.cc:1918] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:04:19.368362 26246 raft_consensus.cc:2258] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.368398 26245 log.cc:1196] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/85881397249146ea99262b41748935c3
I20260620 23:04:19.368613 26246 ts_tablet_manager.cc:1905] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.368628 26245 ts_tablet_manager.cc:1939] T 85881397249146ea99262b41748935c3 P 927abf2707534570b8ee54b52872516c: Deleting consensus metadata
I20260620 23:04:19.368932 25701 catalog_manager.cc:4841] TS 927abf2707534570b8ee54b52872516c (127.17.188.67:39015): tablet 85881397249146ea99262b41748935c3 (table test-table [id=7cd2e0e7bf5b4e33a086254627e3b5af]) successfully deleted
I20260620 23:04:19.369041 26247 ts_tablet_manager.cc:1905] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.370149 26246 ts_tablet_manager.cc:1918] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:04:19.370204 26246 log.cc:1196] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/85881397249146ea99262b41748935c3
I20260620 23:04:19.370419 26246 ts_tablet_manager.cc:1939] T 85881397249146ea99262b41748935c3 P da8933fda0cb410684faff151335307b: Deleting consensus metadata
I20260620 23:04:19.370667 26247 ts_tablet_manager.cc:1918] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:04:19.370733 26247 log.cc:1196] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/85881397249146ea99262b41748935c3
I20260620 23:04:19.370949 26247 ts_tablet_manager.cc:1939] T 85881397249146ea99262b41748935c3 P f68c01abea2b46c1b6555f19496ec43c: Deleting consensus metadata
I20260620 23:04:19.371275 25699 catalog_manager.cc:4841] TS f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731): tablet 85881397249146ea99262b41748935c3 (table test-table [id=7cd2e0e7bf5b4e33a086254627e3b5af]) successfully deleted
I20260620 23:04:19.371366 25699 catalog_manager.cc:4841] TS da8933fda0cb410684faff151335307b (127.17.188.65:46681): tablet 85881397249146ea99262b41748935c3 (table test-table [id=7cd2e0e7bf5b4e33a086254627e3b5af]) successfully deleted
Jun 20 23:04:19 dist-test-slave-zpss krb5kdc[25669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:04:19 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:19 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 23:04:19.390054 25763 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:19.384139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.17.188.126:34913)
0620 23:04:19.384202 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.384205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.384297 (+ 92us) server_negotiation.cc:407] Connection header received
0620 23:04:19.384345 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.384346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.384356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.384430 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:19.384971 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.385755 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.386559 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.386833 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.386933 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.389973 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45834: BlockingRecv error: recv got EOF from 127.0.0.1:45834 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 20 23:04:19 dist-test-slave-zpss krb5kdc[25669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM
I20260620 23:04:19.397729 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.390844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45848 (local address 127.17.188.126:34913)
0620 23:04:19.390885 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.390887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.390988 (+ 101us) server_negotiation.cc:407] Connection header received
0620 23:04:19.391060 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.391061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.391070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.391127 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:19.391556 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.391993 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.392634 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.392809 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.392876 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.395086 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:19.395088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:19.395088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:19.395094 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:19.396549 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:19.396945 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:19.396946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:19.396947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:19.397001 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:19.397211 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:19.397212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:19.397212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:19.397382 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:19.397464 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.397598 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.397658 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:04:19.399168 25713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:04:19.399312 25713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:04:19.400619 25991 tablet_service.cc:1467] Processing CreateTablet for tablet 00f6ad45234348f1bea119c9831c5cc9 (DEFAULT_TABLE table=test-table [id=35a50479f13c42e091d4d5a2415425d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:19.400619 25850 tablet_service.cc:1467] Processing CreateTablet for tablet 00f6ad45234348f1bea119c9831c5cc9 (DEFAULT_TABLE table=test-table [id=35a50479f13c42e091d4d5a2415425d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:19.400663 26131 tablet_service.cc:1467] Processing CreateTablet for tablet 00f6ad45234348f1bea119c9831c5cc9 (DEFAULT_TABLE table=test-table [id=35a50479f13c42e091d4d5a2415425d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:19.400746 25991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00f6ad45234348f1bea119c9831c5cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.400746 25850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00f6ad45234348f1bea119c9831c5cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.400781 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00f6ad45234348f1bea119c9831c5cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.402120 26226 tablet_bootstrap.cc:492] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: Bootstrap starting.
I20260620 23:04:19.402173 26222 tablet_bootstrap.cc:492] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: Bootstrap starting.
I20260620 23:04:19.402195 26223 tablet_bootstrap.cc:492] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: Bootstrap starting.
I20260620 23:04:19.402464 26223 tablet_bootstrap.cc:654] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.402539 26226 tablet_bootstrap.cc:654] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.402547 26222 tablet_bootstrap.cc:654] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.402976 26223 tablet_bootstrap.cc:492] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: No bootstrap required, opened a new log
I20260620 23:04:19.403033 26223 ts_tablet_manager.cc:1397] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 23:04:19.403133 26222 tablet_bootstrap.cc:492] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: No bootstrap required, opened a new log
I20260620 23:04:19.403133 26223 raft_consensus.cc:348] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.403188 26223 raft_consensus.cc:374] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.403203 26223 raft_consensus.cc:729] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8933fda0cb410684faff151335307b, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.403192 26222 ts_tablet_manager.cc:1397] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:19.403255 26223 consensus_queue.cc:260] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.403340 26223 ts_tablet_manager.cc:1428] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:19.403337 26226 tablet_bootstrap.cc:492] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: No bootstrap required, opened a new log
I20260620 23:04:19.403378 26226 ts_tablet_manager.cc:1397] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:19.403347 26222 raft_consensus.cc:348] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.403414 26222 raft_consensus.cc:374] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.403453 26222 raft_consensus.cc:729] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f68c01abea2b46c1b6555f19496ec43c, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.403503 26222 consensus_queue.cc:260] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.403513 26226 raft_consensus.cc:348] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.403573 26226 raft_consensus.cc:374] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.403596 26226 raft_consensus.cc:729] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 927abf2707534570b8ee54b52872516c, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.403596 26222 ts_tablet_manager.cc:1428] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:19.403649 26226 consensus_queue.cc:260] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.403734 26226 ts_tablet_manager.cc:1428] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:19.752647 26229 raft_consensus.cc:482] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:04:19.752740 26229 raft_consensus.cc:504] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.753046 26229 leader_election.cc:288] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da8933fda0cb410684faff151335307b (127.17.188.65:46681), 927abf2707534570b8ee54b52872516c (127.17.188.67:39015)
I20260620 23:04:19.757040 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.753277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:34601 (local address 127.17.188.65:46681)
0620 23:04:19.753362 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.753364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.753464 (+ 100us) server_negotiation.cc:407] Connection header received
0620 23:04:19.753523 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.753524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.753536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.753607 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:19.753819 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.754613 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.756129 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.756669 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.756750 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.756762 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.756931 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.756971 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:04:19.757048 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.753377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:48989 (local address 127.17.188.67:39015)
0620 23:04:19.753514 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.753516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.753523 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:04:19.753547 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.753548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.753558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.753620 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:19.753796 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.754613 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.756128 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.756694 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.756759 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.756766 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.756929 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.756969 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260620 23:04:19.757064 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.753225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:46681 (local address 127.17.188.66:34601)
0620 23:04:19.753379 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:19.753391 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:19.753455 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:19.753677 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:19.753677 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:19.753678 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:19.753715 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.753716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.754707 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.754708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.756043 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.756045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.756783 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.756785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.756857 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.756859 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:04:19.756912 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 23:04:19.756960 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":68,"thread_start_us":67,"threads_started":1}
I20260620 23:04:19.757076 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.753243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:39015 (local address 127.17.188.66:48989)
0620 23:04:19.753391 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:19.753400 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:19.753453 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:19.753639 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:19.753640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:19.753643 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:19.753704 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.753706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.754687 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.754688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.756043 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.756045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.756784 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.756785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.756850 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.756855 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:04:19.756912 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:04:19.756960 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":32,"thread_start_us":58,"threads_started":1}
I20260620 23:04:19.757337 25871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00f6ad45234348f1bea119c9831c5cc9" candidate_uuid: "f68c01abea2b46c1b6555f19496ec43c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da8933fda0cb410684faff151335307b" is_pre_election: true
I20260620 23:04:19.757381 26151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00f6ad45234348f1bea119c9831c5cc9" candidate_uuid: "f68c01abea2b46c1b6555f19496ec43c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "927abf2707534570b8ee54b52872516c" is_pre_election: true
I20260620 23:04:19.757455 26151 raft_consensus.cc:2454] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f68c01abea2b46c1b6555f19496ec43c in term 0.
I20260620 23:04:19.757489 25871 raft_consensus.cc:2454] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f68c01abea2b46c1b6555f19496ec43c in term 0.
I20260620 23:04:19.757638 25945 leader_election.cc:302] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 927abf2707534570b8ee54b52872516c, f68c01abea2b46c1b6555f19496ec43c; no voters:
I20260620 23:04:19.757791 26229 raft_consensus.cc:2781] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:04:19.757833 26229 raft_consensus.cc:482] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:04:19.757853 26229 raft_consensus.cc:3037] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.758508 26229 raft_consensus.cc:504] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.758635 26229 leader_election.cc:288] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [CANDIDATE]: Term 1 election: Requested vote from peers da8933fda0cb410684faff151335307b (127.17.188.65:46681), 927abf2707534570b8ee54b52872516c (127.17.188.67:39015)
I20260620 23:04:19.758781 26151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00f6ad45234348f1bea119c9831c5cc9" candidate_uuid: "f68c01abea2b46c1b6555f19496ec43c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "927abf2707534570b8ee54b52872516c"
I20260620 23:04:19.758806 25871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00f6ad45234348f1bea119c9831c5cc9" candidate_uuid: "f68c01abea2b46c1b6555f19496ec43c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da8933fda0cb410684faff151335307b"
I20260620 23:04:19.758855 26151 raft_consensus.cc:3037] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.758862 25871 raft_consensus.cc:3037] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.759560 26151 raft_consensus.cc:2454] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f68c01abea2b46c1b6555f19496ec43c in term 1.
I20260620 23:04:19.759658 25871 raft_consensus.cc:2454] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f68c01abea2b46c1b6555f19496ec43c in term 1.
I20260620 23:04:19.759728 25945 leader_election.cc:302] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 927abf2707534570b8ee54b52872516c, f68c01abea2b46c1b6555f19496ec43c; no voters:
I20260620 23:04:19.759816 26229 raft_consensus.cc:2781] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:19.759972 26229 raft_consensus.cc:686] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 1 LEADER]: Becoming Leader. State: Replica: f68c01abea2b46c1b6555f19496ec43c, State: Running, Role: LEADER
I20260620 23:04:19.760071 26229 consensus_queue.cc:237] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.760700 25713 catalog_manager.cc:5495] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c reported cstate change: term changed from 0 to 1, leader changed from <none> to f68c01abea2b46c1b6555f19496ec43c (127.17.188.66). New cstate: current_term: 1 leader_uuid: "f68c01abea2b46c1b6555f19496ec43c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } health_report { overall_health: UNKNOWN } } }
W20260620 23:04:19.842621 26192 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.17.188.67:39015
I20260620 23:04:19.845187 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.842894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40488 (local address 127.17.188.66:39731)
0620 23:04:19.843030 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.843032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.843041 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:19.843082 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.843083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.843100 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.843169 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:19.843309 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.843797 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.844517 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.844709 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.844762 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.844889 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.844980 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.845028 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.845064 (+ 36us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.845079 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":84,"threads_started":1}
I20260620 23:04:19.846097 26151 raft_consensus.cc:1261] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Refusing update from remote peer f68c01abea2b46c1b6555f19496ec43c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:19.846164 25871 raft_consensus.cc:1261] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Refusing update from remote peer f68c01abea2b46c1b6555f19496ec43c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:19.846329 26229 consensus_queue.cc:1035] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [LEADER]: Connected to new peer: Peer: permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:19.846449 26292 consensus_queue.cc:1035] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [LEADER]: Connected to new peer: Peer: permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:19.849746 25713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:19.849841 25713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:19.850391 25713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943: Sending DeleteTablet for 3 replicas of tablet 00f6ad45234348f1bea119c9831c5cc9
I20260620 23:04:19.850618 26131 tablet_service.cc:1514] Processing DeleteTablet for tablet 00f6ad45234348f1bea119c9831c5cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:19 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56044
I20260620 23:04:19.850636 25991 tablet_service.cc:1514] Processing DeleteTablet for tablet 00f6ad45234348f1bea119c9831c5cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:19 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:40480
I20260620 23:04:19.850669 25850 tablet_service.cc:1514] Processing DeleteTablet for tablet 00f6ad45234348f1bea119c9831c5cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:19 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43290
I20260620 23:04:19.850719 26247 tablet_replica.cc:330] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: stopping tablet replica
I20260620 23:04:19.850757 26245 tablet_replica.cc:330] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: stopping tablet replica
I20260620 23:04:19.850768 26247 raft_consensus.cc:2229] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:04:19.850804 26245 raft_consensus.cc:2229] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:19.850842 26247 raft_consensus.cc:2258] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.850855 26245 raft_consensus.cc:2258] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.850921 26246 tablet_replica.cc:330] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: stopping tablet replica
I20260620 23:04:19.850977 26246 raft_consensus.cc:2229] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:19.851022 26246 raft_consensus.cc:2258] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.851135 26247 ts_tablet_manager.cc:1905] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.851522 26246 ts_tablet_manager.cc:1905] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.851577 25713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:04:19.851706 25713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:04:19.852689 26247 ts_tablet_manager.cc:1918] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:19.852736 26247 log.cc:1196] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/00f6ad45234348f1bea119c9831c5cc9
I20260620 23:04:19.852942 26247 ts_tablet_manager.cc:1939] T 00f6ad45234348f1bea119c9831c5cc9 P f68c01abea2b46c1b6555f19496ec43c: Deleting consensus metadata
I20260620 23:04:19.853139 26246 ts_tablet_manager.cc:1918] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:19.853184 26246 log.cc:1196] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/00f6ad45234348f1bea119c9831c5cc9
I20260620 23:04:19.853247 25699 catalog_manager.cc:4841] TS f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731): tablet 00f6ad45234348f1bea119c9831c5cc9 (table test-table [id=35a50479f13c42e091d4d5a2415425d9]) successfully deleted
I20260620 23:04:19.853451 26246 ts_tablet_manager.cc:1939] T 00f6ad45234348f1bea119c9831c5cc9 P da8933fda0cb410684faff151335307b: Deleting consensus metadata
I20260620 23:04:19.853777 25699 catalog_manager.cc:4841] TS da8933fda0cb410684faff151335307b (127.17.188.65:46681): tablet 00f6ad45234348f1bea119c9831c5cc9 (table test-table [id=35a50479f13c42e091d4d5a2415425d9]) successfully deleted
I20260620 23:04:19.854017 26245 ts_tablet_manager.cc:1905] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.854483 26131 tablet_service.cc:1467] Processing CreateTablet for tablet e4dcd5a131ac4732965bfa2fbeacdd62 (DEFAULT_TABLE table=test-table [id=57fc1c3ffa8444d6aa696540fd170997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:19.854583 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4dcd5a131ac4732965bfa2fbeacdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.854617 25991 tablet_service.cc:1467] Processing CreateTablet for tablet e4dcd5a131ac4732965bfa2fbeacdd62 (DEFAULT_TABLE table=test-table [id=57fc1c3ffa8444d6aa696540fd170997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:19.854728 25991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4dcd5a131ac4732965bfa2fbeacdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.855551 26245 ts_tablet_manager.cc:1918] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:19.855594 26245 log.cc:1196] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/00f6ad45234348f1bea119c9831c5cc9
I20260620 23:04:19.855793 26226 tablet_bootstrap.cc:492] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: Bootstrap starting.
I20260620 23:04:19.855939 26222 tablet_bootstrap.cc:492] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: Bootstrap starting.
I20260620 23:04:19.855959 25850 tablet_service.cc:1467] Processing CreateTablet for tablet e4dcd5a131ac4732965bfa2fbeacdd62 (DEFAULT_TABLE table=test-table [id=57fc1c3ffa8444d6aa696540fd170997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:19.856038 25850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4dcd5a131ac4732965bfa2fbeacdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.856173 26226 tablet_bootstrap.cc:654] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.856298 26222 tablet_bootstrap.cc:654] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.857069 26222 tablet_bootstrap.cc:492] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: No bootstrap required, opened a new log
I20260620 23:04:19.857107 26222 ts_tablet_manager.cc:1397] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:19.857249 26223 tablet_bootstrap.cc:492] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: Bootstrap starting.
I20260620 23:04:19.857255 26222 raft_consensus.cc:348] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.855793 26245 ts_tablet_manager.cc:1939] T 00f6ad45234348f1bea119c9831c5cc9 P 927abf2707534570b8ee54b52872516c: Deleting consensus metadata
I20260620 23:04:19.857525 26222 raft_consensus.cc:374] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.857559 26222 raft_consensus.cc:729] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f68c01abea2b46c1b6555f19496ec43c, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.857558 26223 tablet_bootstrap.cc:654] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.857600 26222 consensus_queue.cc:260] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.857692 26222 ts_tablet_manager.cc:1428] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 23:04:19.858008 26223 tablet_bootstrap.cc:492] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: No bootstrap required, opened a new log
I20260620 23:04:19.858012 25701 catalog_manager.cc:4841] TS 927abf2707534570b8ee54b52872516c (127.17.188.67:39015): tablet 00f6ad45234348f1bea119c9831c5cc9 (table test-table [id=35a50479f13c42e091d4d5a2415425d9]) successfully deleted
I20260620 23:04:19.858067 26223 ts_tablet_manager.cc:1397] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:19.858117 26226 tablet_bootstrap.cc:492] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: No bootstrap required, opened a new log
I20260620 23:04:19.858148 26226 ts_tablet_manager.cc:1397] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:04:19.858153 26223 raft_consensus.cc:348] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.858237 26223 raft_consensus.cc:374] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.858255 26223 raft_consensus.cc:729] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8933fda0cb410684faff151335307b, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.858294 26223 consensus_queue.cc:260] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.858283 26226 raft_consensus.cc:348] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.858340 26226 raft_consensus.cc:374] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.858364 26226 raft_consensus.cc:729] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 927abf2707534570b8ee54b52872516c, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.858393 26223 ts_tablet_manager.cc:1428] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:19.858400 26226 consensus_queue.cc:260] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.858484 26226 ts_tablet_manager.cc:1428] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:19.932178 26230 raft_consensus.cc:482] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:04:19.932292 26230 raft_consensus.cc:504] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.932634 26230 leader_election.cc:288] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da8933fda0cb410684faff151335307b (127.17.188.65:46681), f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731)
I20260620 23:04:19.936146 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.932837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:46681 (local address 127.17.188.67:52355)
0620 23:04:19.932977 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:19.932987 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:19.933071 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:19.933270 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:19.933271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:19.933274 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:19.933377 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.933379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.934323 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.934323 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.935077 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.935079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.935867 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.935868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.935935 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.935940 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:04:19.936019 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:04:19.936060 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260620 23:04:19.936136 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.932899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:52355 (local address 127.17.188.65:46681)
0620 23:04:19.932961 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.932963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.933041 (+ 78us) server_negotiation.cc:407] Connection header received
0620 23:04:19.933161 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.933162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.933173 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.933205 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:19.933457 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.934233 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.935209 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.935780 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.935852 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.935861 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.936031 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.936076 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:04:19.936411 25871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4dcd5a131ac4732965bfa2fbeacdd62" candidate_uuid: "927abf2707534570b8ee54b52872516c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da8933fda0cb410684faff151335307b" is_pre_election: true
I20260620 23:04:19.936527 25871 raft_consensus.cc:2454] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 927abf2707534570b8ee54b52872516c in term 0.
I20260620 23:04:19.936690 26083 leader_election.cc:302] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 927abf2707534570b8ee54b52872516c, da8933fda0cb410684faff151335307b; no voters:
I20260620 23:04:19.936805 26230 raft_consensus.cc:2781] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:04:19.936837 26230 raft_consensus.cc:482] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:04:19.936864 26230 raft_consensus.cc:3037] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.937120 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.933028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:39731 (local address 127.17.188.67:33641)
0620 23:04:19.933114 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:19.933124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:19.933168 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:19.933309 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:19.933326 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:19.933328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:19.933380 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.933381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.935146 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.935148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.935910 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.935912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.936867 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.936868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.936944 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.936948 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:04:19.937005 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:04:19.937044 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":44,"threads_started":1}
I20260620 23:04:19.937129 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.933086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:33641 (local address 127.17.188.66:39731)
0620 23:04:19.933123 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.933125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.933160 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:04:19.933234 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.933235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.933244 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.933309 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:19.934344 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.935059 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.935992 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.936745 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.936823 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.936831 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.937018 (+ 187us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.937057 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:04:19.937341 26011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4dcd5a131ac4732965bfa2fbeacdd62" candidate_uuid: "927abf2707534570b8ee54b52872516c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68c01abea2b46c1b6555f19496ec43c" is_pre_election: true
I20260620 23:04:19.937418 26011 raft_consensus.cc:2454] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 927abf2707534570b8ee54b52872516c in term 0.
I20260620 23:04:19.937753 26230 raft_consensus.cc:504] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.937875 26230 leader_election.cc:288] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [CANDIDATE]: Term 1 election: Requested vote from peers da8933fda0cb410684faff151335307b (127.17.188.65:46681), f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731)
I20260620 23:04:19.938010 25871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4dcd5a131ac4732965bfa2fbeacdd62" candidate_uuid: "927abf2707534570b8ee54b52872516c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da8933fda0cb410684faff151335307b"
I20260620 23:04:19.938086 25871 raft_consensus.cc:3037] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.938095 26011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4dcd5a131ac4732965bfa2fbeacdd62" candidate_uuid: "927abf2707534570b8ee54b52872516c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68c01abea2b46c1b6555f19496ec43c"
I20260620 23:04:19.938153 26011 raft_consensus.cc:3037] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.938803 25871 raft_consensus.cc:2454] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 927abf2707534570b8ee54b52872516c in term 1.
I20260620 23:04:19.938803 26011 raft_consensus.cc:2454] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 927abf2707534570b8ee54b52872516c in term 1.
I20260620 23:04:19.938963 26083 leader_election.cc:302] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 927abf2707534570b8ee54b52872516c, da8933fda0cb410684faff151335307b; no voters:
I20260620 23:04:19.939051 26230 raft_consensus.cc:2781] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:19.939195 26230 raft_consensus.cc:686] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 1 LEADER]: Becoming Leader. State: Replica: 927abf2707534570b8ee54b52872516c, State: Running, Role: LEADER
I20260620 23:04:19.939291 26230 consensus_queue.cc:237] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } }
I20260620 23:04:19.940066 25714 catalog_manager.cc:5495] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c reported cstate change: term changed from 0 to 1, leader changed from <none> to 927abf2707534570b8ee54b52872516c (127.17.188.67). New cstate: current_term: 1 leader_uuid: "927abf2707534570b8ee54b52872516c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "927abf2707534570b8ee54b52872516c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39015 } health_report { overall_health: HEALTHY } } }
I20260620 23:04:19.952625 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.950191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.17.188.67:39015)
0620 23:04:19.950243 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.950245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.950252 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:04:19.950284 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.950285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.950303 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.950373 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:19.950466 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.951180 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.951849 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.952153 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.952224 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.952326 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.952446 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.952512 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.952534 (+ 22us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.952553 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:04:19.957193 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.954863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:52035 (local address 127.17.188.65:46681)
0620 23:04:19.954902 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.954904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.955026 (+ 122us) server_negotiation.cc:407] Connection header received
0620 23:04:19.955099 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.955100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.955110 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.955163 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:19.955294 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.955741 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.956483 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.956696 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.956776 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.956942 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.956975 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.957039 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.957099 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.957120 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:04:19.957217 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.954830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:46681 (local address 127.17.188.67:52035)
0620 23:04:19.954957 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:19.954967 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:19.955041 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:19.955166 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:19.955166 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:19.955168 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:04:19.955229 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.955231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.955826 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.955827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.956402 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.956404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.956778 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.956779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.956841 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.956859 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.957040 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.957042 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:04:19.957095 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 23:04:19.957135 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
I20260620 23:04:19.959080 26011 raft_consensus.cc:1261] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Refusing update from remote peer 927abf2707534570b8ee54b52872516c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:19.959111 25871 raft_consensus.cc:1261] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Refusing update from remote peer 927abf2707534570b8ee54b52872516c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:19.959335 26304 consensus_queue.cc:1035] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [LEADER]: Connected to new peer: Peer: permanent_uuid: "da8933fda0cb410684faff151335307b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:19.959471 26230 consensus_queue.cc:1035] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f68c01abea2b46c1b6555f19496ec43c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:19.967031 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.964445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:39015 (local address 127.17.188.65:46929)
0620 23:04:19.964565 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:19.964574 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:19.964634 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:19.964813 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:19.964814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:19.964815 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:04:19.964852 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.964853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.965583 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.965583 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.966390 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:19.966393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.966771 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.966771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:19.966807 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.966810 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.966921 (+ 111us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.966922 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:04:19.966946 (+ 24us) client_negotiation.cc:240] Negotiation successful
0620 23:04:19.966978 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":80,"threads_started":1}
I20260620 23:04:19.967033 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.964625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:46929 (local address 127.17.188.67:39015)
0620 23:04:19.964673 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.964675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.964684 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:19.964708 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.964709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.964722 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.964812 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:19.964892 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.965506 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.966450 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:19.966705 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:19.966775 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:19.966857 (+ 82us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:19.966883 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:19.966923 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.966958 (+ 35us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.966975 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:04:19.976001 25713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:19.976089 25713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:19.976670 25713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1619ed2cc7f4b87be05697f82d37943: Sending DeleteTablet for 3 replicas of tablet e4dcd5a131ac4732965bfa2fbeacdd62
I20260620 23:04:19.976909 26131 tablet_service.cc:1514] Processing DeleteTablet for tablet e4dcd5a131ac4732965bfa2fbeacdd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:19 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:56044
I20260620 23:04:19.976948 25991 tablet_service.cc:1514] Processing DeleteTablet for tablet e4dcd5a131ac4732965bfa2fbeacdd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:19 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:40480
I20260620 23:04:19.976994 26245 tablet_replica.cc:330] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: stopping tablet replica
I20260620 23:04:19.977038 26245 raft_consensus.cc:2229] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:04:19.977043 26247 tablet_replica.cc:330] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: stopping tablet replica
I20260620 23:04:19.977093 26247 raft_consensus.cc:2229] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:19.977123 26247 raft_consensus.cc:2258] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.977120 26245 raft_consensus.cc:2258] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.977265 25850 tablet_service.cc:1514] Processing DeleteTablet for tablet e4dcd5a131ac4732965bfa2fbeacdd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:19 UTC) from {username='oryx', principal='oryx/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43290
I20260620 23:04:19.977370 26246 tablet_replica.cc:330] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: stopping tablet replica
I20260620 23:04:19.977418 26246 raft_consensus.cc:2229] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:19.977444 26246 raft_consensus.cc:2258] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:19.977504 26247 ts_tablet_manager.cc:1905] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.977699 26245 ts_tablet_manager.cc:1905] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.978752 26246 ts_tablet_manager.cc:1905] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:19.979080 26245 ts_tablet_manager.cc:1918] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:04:19.979085 26247 ts_tablet_manager.cc:1918] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:04:19.979137 26245 log.cc:1196] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/e4dcd5a131ac4732965bfa2fbeacdd62
I20260620 23:04:19.979137 26247 log.cc:1196] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/e4dcd5a131ac4732965bfa2fbeacdd62
I20260620 23:04:19.979377 26247 ts_tablet_manager.cc:1939] T e4dcd5a131ac4732965bfa2fbeacdd62 P f68c01abea2b46c1b6555f19496ec43c: Deleting consensus metadata
I20260620 23:04:19.979379 26245 ts_tablet_manager.cc:1939] T e4dcd5a131ac4732965bfa2fbeacdd62 P 927abf2707534570b8ee54b52872516c: Deleting consensus metadata
I20260620 23:04:19.979661 25699 catalog_manager.cc:4841] TS f68c01abea2b46c1b6555f19496ec43c (127.17.188.66:39731): tablet e4dcd5a131ac4732965bfa2fbeacdd62 (table test-table [id=57fc1c3ffa8444d6aa696540fd170997]) successfully deleted
I20260620 23:04:19.980065 26246 ts_tablet_manager.cc:1918] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:04:19.980111 26246 log.cc:1196] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/e4dcd5a131ac4732965bfa2fbeacdd62
I20260620 23:04:19.980254 26246 ts_tablet_manager.cc:1939] T e4dcd5a131ac4732965bfa2fbeacdd62 P da8933fda0cb410684faff151335307b: Deleting consensus metadata
I20260620 23:04:19.980429 25701 catalog_manager.cc:4841] TS 927abf2707534570b8ee54b52872516c (127.17.188.67:39015): tablet e4dcd5a131ac4732965bfa2fbeacdd62 (table test-table [id=57fc1c3ffa8444d6aa696540fd170997]) successfully deleted
I20260620 23:04:19.980710 25699 catalog_manager.cc:4841] TS da8933fda0cb410684faff151335307b (127.17.188.65:46681): tablet e4dcd5a131ac4732965bfa2fbeacdd62 (table test-table [id=57fc1c3ffa8444d6aa696540fd170997]) successfully deleted
I20260620 23:04:19.980792 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 25786
I20260620 23:04:19.988227 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 25926
I20260620 23:04:19.993637 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 26066
I20260620 23:04:19.998205 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 25682
2026-06-20T23:04:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2923 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:20 dist-test-slave-zpss krb5kdc[26320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:20 dist-test-slave-zpss krb5kdc[26320](info): set up 2 sockets
Jun 20 23:04:20 dist-test-slave-zpss krb5kdc[26320](info): commencing operation
krb5kdc: starting...
W20260620 23:04:21.021744 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:04:21 dist-test-slave-zpss krb5kdc[26320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:04:21.130211 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:46093
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:46093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:21.207227 26333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:21.207408 26333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:21.207429 26333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:21.208530 26333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:21.208564 26333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:21.208578 26333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:21.208591 26333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:21.208603 26333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:21.209972 26333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:46093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:46093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:21.210137 26333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:21.210294 26333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:04:21.214870 26333 server_base.cc:1034] running on GCE node
W20260620 23:04:21.214900 26338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:21.214984 26341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:21.215075 26339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:21.215391 26333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:21.215663 26333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:21.216831 26333 hybrid_clock.cc:648] HybridClock initialized: now 1781996661216808 us; error 47 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T23:04:21Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1228 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:21 dist-test-slave-zpss krb5kdc[26346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:21 dist-test-slave-zpss krb5kdc[26346](info): set up 2 sockets
Jun 20 23:04:21 dist-test-slave-zpss krb5kdc[26346](info): commencing operation
krb5kdc: starting...
W20260620 23:04:22.242040 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:04:22 dist-test-slave-zpss krb5kdc[26346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:04:22.354203 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35593
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:35593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:22.432382 26359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:22.432559 26359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:22.432579 26359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:22.433723 26359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:22.433761 26359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:22.433776 26359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:22.433789 26359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:22.433800 26359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:22.435246 26359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:35593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:22.435468 26359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:22.435667 26359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:22.440163 26365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:22.440234 26367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:22.440287 26364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:22.440397 26359 server_base.cc:1034] running on GCE node
I20260620 23:04:22.440639 26359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:22.440954 26359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:22.442102 26359 hybrid_clock.cc:648] HybridClock initialized: now 1781996662442092 us; error 41 us; skew 500 ppm
Jun 20 23:04:22 dist-test-slave-zpss krb5kdc[26346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:22.444360 26359 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:04:22.444741 26359 webserver.cc:460] Webserver started at http://127.17.188.126:42015/ using document root <none> and password file <none>
I20260620 23:04:22.444959 26359 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:22.445021 26359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:22.445158 26359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:22.446257 26359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "3a0dfc1aca644ca092c20ce0c3241e3f"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "368649da9ffa7e0d0ecd09c1d24417e1"
server_key_iv: "69cc3fe31ffe1e436e02ecf6bb66b1bc"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.446592 26359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "3a0dfc1aca644ca092c20ce0c3241e3f"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "368649da9ffa7e0d0ecd09c1d24417e1"
server_key_iv: "69cc3fe31ffe1e436e02ecf6bb66b1bc"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.447913 26359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:04:22.448580 26374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.448734 26359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:22.448827 26359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "3a0dfc1aca644ca092c20ce0c3241e3f"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "368649da9ffa7e0d0ecd09c1d24417e1"
server_key_iv: "69cc3fe31ffe1e436e02ecf6bb66b1bc"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.448902 26359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:22.470309 26359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:22.471977 26359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:22.472124 26359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:22.475689 26359 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:35593
I20260620 23:04:22.475723 26426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:35593 every 8 connection(s)
I20260620 23:04:22.476037 26359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:22.476838 26427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:22.478283 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 26359
I20260620 23:04:22.478396 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:04:22.478564 18161 external_mini_cluster.cc:1428] Setting key 1cac63f0b5d0542724e723ebf86e3dcb
I20260620 23:04:22.479481 26427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: Bootstrap starting.
I20260620 23:04:22.480139 26427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:22.480432 26427 log.cc:824] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:22.481148 26427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: No bootstrap required, opened a new log
I20260620 23:04:22.483428 26427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } }
I20260620 23:04:22.483608 26427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:22.483645 26427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a0dfc1aca644ca092c20ce0c3241e3f, State: Initialized, Role: FOLLOWER
I20260620 23:04:22.483815 26427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } }
I20260620 23:04:22.483913 26427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:22.483958 26427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:22.484007 26427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:22.484548 26427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } }
I20260620 23:04:22.484660 26427 leader_election.cc:302] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a0dfc1aca644ca092c20ce0c3241e3f; no voters:
I20260620 23:04:22.484819 26427 leader_election.cc:288] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:22.484853 26432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:22.484966 26432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 1 LEADER]: Becoming Leader. State: Replica: 3a0dfc1aca644ca092c20ce0c3241e3f, State: Running, Role: LEADER
I20260620 23:04:22.485083 26427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:22.485112 26432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } }
Jun 20 23:04:22 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:04:22.485579 26433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } } }
I20260620 23:04:22.485592 26434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a0dfc1aca644ca092c20ce0c3241e3f. Latest consensus state: current_term: 1 leader_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } } }
I20260620 23:04:22.485680 26433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:22.485694 26434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: This master's current role is: LEADER
W20260620 23:04:22.486999 26448 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:04:22.487056 26448 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:04:22.487105 26439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:22.487255 26439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:22.488756 26439 catalog_manager.cc:1347] Generated new cluster ID: 18a2a5060559424ea81b6ce7d5289559
I20260620 23:04:22.488806 26439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:22.489854 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:22.479076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.17.188.126:35593)
0620 23:04:22.479252 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:22.479261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:22.479290 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:04:22.479851 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:22.479861 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:22.479920 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:22.480069 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:22.480711 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.482068 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.482805 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.483497 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.483610 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:22.486269 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:22.486273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:22.486273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:22.486286 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:22.488114 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:22.488510 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:22.488513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:22.488513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:22.488611 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:22.488902 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:22.488903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:22.488904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:22.489190 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:22.489363 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:22.489519 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 23:04:22.489654 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260620 23:04:22.506045 26439 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:22.506525 26439 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:22.510677 26439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: Generated new TSK 0
I20260620 23:04:22.510840 26439 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:04:22.539772 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35593
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:22.623019 26453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:22.623200 26453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:22.623225 26453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:22.624436 26453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:22.624482 26453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:22.624511 26453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:04:22.626000 26453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:22.626257 26453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:22.626474 26453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:22.630940 26458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:22.630934 26459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:22.630934 26461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:22.631033 26453 server_base.cc:1034] running on GCE node
I20260620 23:04:22.631316 26453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:22.631572 26453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:22.632728 26453 hybrid_clock.cc:648] HybridClock initialized: now 1781996662632718 us; error 34 us; skew 500 ppm
Jun 20 23:04:22 dist-test-slave-zpss krb5kdc[26346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:22.635200 26453 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:04:22.635602 26453 webserver.cc:460] Webserver started at http://127.17.188.65:36001/ using document root <none> and password file <none>
I20260620 23:04:22.635871 26453 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:22.635937 26453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:22.636094 26453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:22.637077 26453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "715665c6a92a4072868a3091c441ca92"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "b356903fc3d26ce6e2840e7f49ead2f1"
server_key_iv: "f04c6c350dce981437f60cb2cf3f4351"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.637476 26453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "715665c6a92a4072868a3091c441ca92"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "b356903fc3d26ce6e2840e7f49ead2f1"
server_key_iv: "f04c6c350dce981437f60cb2cf3f4351"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.638707 26453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:04:22.639406 26468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.639568 26453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:04:22.639662 26453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "715665c6a92a4072868a3091c441ca92"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "b356903fc3d26ce6e2840e7f49ead2f1"
server_key_iv: "f04c6c350dce981437f60cb2cf3f4351"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.639740 26453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:22.655917 26453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:22.657755 26453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:22.657909 26453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:22.658129 26453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:22.658430 26453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:22.658485 26453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.658521 26453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:22.658548 26453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.664570 26453 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:32937
I20260620 23:04:22.664602 26581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:32937 every 8 connection(s)
I20260620 23:04:22.664954 26453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:04:22 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:04:22.673538 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:22.665655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:42681 (local address 127.17.188.126:35593)
0620 23:04:22.665697 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:22.665699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:22.665868 (+ 169us) server_negotiation.cc:407] Connection header received
0620 23:04:22.666349 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:22.666351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:22.666363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:22.666422 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:22.667227 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.667692 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.668582 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.668791 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.668866 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:22.670894 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:22.670896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:22.670896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:22.670904 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:22.672217 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:22.672589 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:22.672590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:22.672590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:22.672647 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:22.672890 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:22.672891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:22.672892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:22.673003 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:22.673081 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:22.673360 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 23:04:22.673448 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:04:22.673681 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:22.665616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:42681)
0620 23:04:22.665750 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:22.665790 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:22.666247 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:22.666479 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:22.666483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:22.666974 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:22.667132 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:22.667136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.667787 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.667788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:22.668499 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:22.668501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.668875 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.668876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:22.668990 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:22.669465 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:22.669472 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:22.670803 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:22.672309 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:22.672316 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:22.672318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:22.672519 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:22.672709 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:22.672710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:22.672710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:22.672827 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:22.673072 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:22.673078 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:22.673207 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 23:04:22.673303 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 23:04:22.673463 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
I20260620 23:04:22.674167 26582 heartbeater.cc:347] Connected to a master server at 127.17.188.126:35593
I20260620 23:04:22.674239 26582 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:22.674337 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 26453
I20260620 23:04:22.674417 26582 heartbeater.cc:511] Master 127.17.188.126:35593 requested a full tablet report, sending...
I20260620 23:04:22.674464 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:04:22.674706 18161 external_mini_cluster.cc:1428] Setting key 997cba15e9f846ccc8ae245563c0f8db
I20260620 23:04:22.674943 26391 ts_manager.cc:194] Registered new tserver with Master: 715665c6a92a4072868a3091c441ca92 (127.17.188.65:32937)
I20260620 23:04:22.675698 26391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:42681
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:04:22.698326 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35593
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:22.774423 26587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:22.774655 26587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:22.774694 26587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:22.775795 26587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:22.775856 26587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:22.775897 26587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:04:22.777225 26587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:22.777496 26587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:22.777712 26587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:22.782272 26592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:22.782272 26593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:22.782330 26587 server_base.cc:1034] running on GCE node
W20260620 23:04:22.782317 26595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:22.782620 26587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:22.782846 26587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:22.783983 26587 hybrid_clock.cc:648] HybridClock initialized: now 1781996662783966 us; error 37 us; skew 500 ppm
Jun 20 23:04:22 dist-test-slave-zpss krb5kdc[26346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:22.786509 26587 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:04:22.786943 26587 webserver.cc:460] Webserver started at http://127.17.188.66:40165/ using document root <none> and password file <none>
I20260620 23:04:22.787151 26587 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:22.787213 26587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:22.787344 26587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:22.788498 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "4080bd858b7b44849457e69631135648"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "85a5355577cf27c62d7f242c8a3916a3"
server_key_iv: "5419d75bdc5ca5d574a9bf3d757ca547"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.788908 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "4080bd858b7b44849457e69631135648"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "85a5355577cf27c62d7f242c8a3916a3"
server_key_iv: "5419d75bdc5ca5d574a9bf3d757ca547"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.790253 26587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:04:22.790902 26602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.791044 26587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.791129 26587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "4080bd858b7b44849457e69631135648"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "85a5355577cf27c62d7f242c8a3916a3"
server_key_iv: "5419d75bdc5ca5d574a9bf3d757ca547"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.791215 26587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:22.811218 26587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:22.812893 26587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:22.813045 26587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:22.813258 26587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:22.813555 26587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:22.813604 26587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.813637 26587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:22.813666 26587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.818873 26587 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:38021
I20260620 23:04:22.818953 26715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:38021 every 8 connection(s)
I20260620 23:04:22.819234 26587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:04:22.822710 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 26587
I20260620 23:04:22.822817 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:04:22.822989 18161 external_mini_cluster.cc:1428] Setting key af8f1f7f5de50dec07550e06a0133c89
Jun 20 23:04:22 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:04:22.829967 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:22.819884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:38239 (local address 127.17.188.126:35593)
0620 23:04:22.819926 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:22.819928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:22.820091 (+ 163us) server_negotiation.cc:407] Connection header received
0620 23:04:22.820557 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:22.820558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:22.820570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:22.820629 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:22.821552 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.822031 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.823366 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.823606 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.823671 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:22.826984 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:22.826985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:22.826986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:22.826994 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:22.828669 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:22.829031 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:22.829032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:22.829033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:22.829113 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:22.829349 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:22.829350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:22.829350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:22.829486 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:22.829564 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:22.829793 (+ 229us) server_negotiation.cc:299] Negotiation successful
0620 23:04:22.829873 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:04:22.829983 26718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:22.819849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:38239)
0620 23:04:22.819971 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:22.820014 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:22.820462 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:22.820692 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:22.820695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:22.821225 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:22.821461 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:22.821465 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.822133 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.822134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:22.823265 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:22.823268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.824838 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.824839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:22.824947 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:22.825441 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:22.825449 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:22.826896 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:22.828748 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:22.828753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:22.828755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:22.828965 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:22.829183 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:22.829183 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:22.829183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:22.829270 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:22.829554 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:22.829558 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:22.829669 (+ 111us) client_negotiation.cc:769] Sending connection context
0620 23:04:22.829746 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 23:04:22.829848 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":59,"threads_started":1}
I20260620 23:04:22.830389 26716 heartbeater.cc:347] Connected to a master server at 127.17.188.126:35593
I20260620 23:04:22.830493 26716 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:22.830667 26716 heartbeater.cc:511] Master 127.17.188.126:35593 requested a full tablet report, sending...
I20260620 23:04:22.831009 26391 ts_manager.cc:194] Registered new tserver with Master: 4080bd858b7b44849457e69631135648 (127.17.188.66:38021)
I20260620 23:04:22.831360 26391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:38239
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:04:22.846040 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35593
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:22.923583 26721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:22.923775 26721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:22.923799 26721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:22.924880 26721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:22.924912 26721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:22.924938 26721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:04:22.926283 26721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:22.926473 26721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:22.926632 26721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:22.931473 26727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:22.931479 26729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:22.931545 26726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:22.931592 26721 server_base.cc:1034] running on GCE node
I20260620 23:04:22.931850 26721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:22.932066 26721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:22.933207 26721 hybrid_clock.cc:648] HybridClock initialized: now 1781996662933189 us; error 36 us; skew 500 ppm
Jun 20 23:04:22 dist-test-slave-zpss krb5kdc[26346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:22.935567 26721 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:04:22.935973 26721 webserver.cc:460] Webserver started at http://127.17.188.67:42283/ using document root <none> and password file <none>
I20260620 23:04:22.936205 26721 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:22.936247 26721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:22.936391 26721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:22.937423 26721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "a0bd6dde23bd4c55b164e3c4108b7d0e"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "1514648ce192740af14170410e1eadc6"
server_key_iv: "a2d8f7fbb0f719301cb42b12f42d30b4"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.937767 26721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "a0bd6dde23bd4c55b164e3c4108b7d0e"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "1514648ce192740af14170410e1eadc6"
server_key_iv: "a2d8f7fbb0f719301cb42b12f42d30b4"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.939028 26721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:22.939790 26736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.939958 26721 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:22.940040 26721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "a0bd6dde23bd4c55b164e3c4108b7d0e"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "1514648ce192740af14170410e1eadc6"
server_key_iv: "a2d8f7fbb0f719301cb42b12f42d30b4"
server_key_version: "encryptionkey@0"
I20260620 23:04:22.940109 26721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:22.963644 26721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:22.965441 26721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:22.965585 26721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:22.965785 26721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:22.966064 26721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:22.966114 26721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.966156 26721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:22.966182 26721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:22.971581 26721 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:41543
I20260620 23:04:22.971621 26849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:41543 every 8 connection(s)
I20260620 23:04:22.971880 26721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:04:22 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:04:22.980294 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 26721
I20260620 23:04:22.980425 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:04:22.980607 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:22.972518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:45675 (local address 127.17.188.126:35593)
0620 23:04:22.972571 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:22.972573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:22.972798 (+ 225us) server_negotiation.cc:407] Connection header received
0620 23:04:22.973376 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:22.973377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:22.973388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:22.973461 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:22.974362 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.974847 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.975801 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.975982 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.976029 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:22.978020 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:22.978021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:22.978022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:22.978046 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:22.979264 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:22.979665 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:22.979666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:22.979666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:22.979712 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:22.979918 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:22.979919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:22.979919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:22.980012 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:22.980061 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:22.980477 (+ 416us) server_negotiation.cc:299] Negotiation successful
0620 23:04:22.980545 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:04:22.980643 18161 external_mini_cluster.cc:1428] Setting key 3f3e4ea6cbb85e20db6b5a6b243487ec
I20260620 23:04:22.980875 26852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:22.972489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:45675)
0620 23:04:22.972663 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:22.972702 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:22.973218 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:22.973522 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:22.973526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:22.974099 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:22.974280 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:22.974286 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.974930 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.974931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:22.975728 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:22.975730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:22.976048 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:22.976049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:22.976153 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:22.976551 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:22.976558 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:22.977935 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:22.979356 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:22.979361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:22.979363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:22.979582 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:22.979768 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:22.979768 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:22.979769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:22.979866 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:22.980067 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:22.980086 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:22.980249 (+ 163us) client_negotiation.cc:769] Sending connection context
0620 23:04:22.980452 (+ 203us) client_negotiation.cc:240] Negotiation successful
0620 23:04:22.980638 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260620 23:04:22.981515 26850 heartbeater.cc:347] Connected to a master server at 127.17.188.126:35593
I20260620 23:04:22.981626 26850 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:22.981796 26850 heartbeater.cc:511] Master 127.17.188.126:35593 requested a full tablet report, sending...
I20260620 23:04:22.982169 26391 ts_manager.cc:194] Registered new tserver with Master: a0bd6dde23bd4c55b164e3c4108b7d0e (127.17.188.67:41543)
I20260620 23:04:22.982574 26391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:45675
I20260620 23:04:22.992312 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab.
Entry for principal oryx/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab.
WARNING: no policy specified for oryx/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab.
Entry for principal oryx/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab.
WARNING: no policy specified for oryx/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab.
Entry for principal oryx/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab.
I20260620 23:04:23.053947 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 26453
I20260620 23:04:23.058382 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 26587
I20260620 23:04:23.064097 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 26721
I20260620 23:04:23.068143 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 26359
I20260620 23:04:23.072732 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35593
--webserver_interface=127.17.188.126
--webserver_port=42015
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:35593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:23.150063 26859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:23.150233 26859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:23.150250 26859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:23.151300 26859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:23.151345 26859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:23.151360 26859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:23.151407 26859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:23.151427 26859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:23.152848 26859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:35593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=42015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:23.153090 26859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:23.153294 26859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:23.157910 26865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:23.157909 26867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:23.157963 26859 server_base.cc:1034] running on GCE node
W20260620 23:04:23.157938 26864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:23.158318 26859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:23.158573 26859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:23.159700 26859 hybrid_clock.cc:648] HybridClock initialized: now 1781996663159681 us; error 35 us; skew 500 ppm
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996663, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:23.162001 26859 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:04:23.162367 26859 webserver.cc:460] Webserver started at http://127.17.188.126:42015/ using document root <none> and password file <none>
I20260620 23:04:23.162562 26859 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:23.162624 26859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:23.163861 26859 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:23.164449 26874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.164595 26859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:23.164690 26859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "3a0dfc1aca644ca092c20ce0c3241e3f"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "368649da9ffa7e0d0ecd09c1d24417e1"
server_key_iv: "69cc3fe31ffe1e436e02ecf6bb66b1bc"
server_key_version: "encryptionkey@0"
I20260620 23:04:23.164988 26859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:23.172828 26859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:23.174503 26859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:23.174646 26859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:23.178306 26859 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:35593
I20260620 23:04:23.178341 26926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:35593 every 8 connection(s)
I20260620 23:04:23.178684 26859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:23.179936 26927 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:04:23.180590 26927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: Bootstrap starting.
I20260620 23:04:23.187431 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 26859
I20260620 23:04:23.187723 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:32937
--local_ip_for_outbound_sockets=127.17.188.65
--tserver_master_addrs=127.17.188.126:35593
--webserver_port=36001
--webserver_interface=127.17.188.65
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.188.65
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:04:23.188480 26927 log.cc:824] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:23.190135 26927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:04:23.190402 26927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: Bootstrap complete.
I20260620 23:04:23.192585 26927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } }
I20260620 23:04:23.192790 26927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a0dfc1aca644ca092c20ce0c3241e3f, State: Initialized, Role: FOLLOWER
I20260620 23:04:23.192939 26927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } }
I20260620 23:04:23.193001 26927 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:23.193065 26927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:23.193109 26927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:04:23.194209 26927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } }
I20260620 23:04:23.194299 26927 leader_election.cc:302] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a0dfc1aca644ca092c20ce0c3241e3f; no voters:
I20260620 23:04:23.194468 26927 leader_election.cc:288] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:04:23.194528 26931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:04:23.194701 26927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:23.194707 26931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [term 2 LEADER]: Becoming Leader. State: Replica: 3a0dfc1aca644ca092c20ce0c3241e3f, State: Running, Role: LEADER
I20260620 23:04:23.194862 26931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } }
I20260620 23:04:23.195171 26932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } } }
I20260620 23:04:23.195276 26932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:23.195169 26931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a0dfc1aca644ca092c20ce0c3241e3f. Latest consensus state: current_term: 2 leader_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a0dfc1aca644ca092c20ce0c3241e3f" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35593 } } }
I20260620 23:04:23.195473 26931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:23.195528 26940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:23.196130 26940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:23.196307 26940 catalog_manager.cc:1259] Loaded cluster ID: 18a2a5060559424ea81b6ce7d5289559
I20260620 23:04:23.196548 26940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:23.197757 26940 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:23.197906 26940 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3a0dfc1aca644ca092c20ce0c3241e3f: Loaded TSK: 0
I20260620 23:04:23.198097 26940 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:04:23.266904 26930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:23.267113 26930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:23.267190 26930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:23.268369 26930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:23.268424 26930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:23.268465 26930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:04:23.269876 26930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.65.keytab
--principal=oryx/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:32937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=36001
--tserver_master_addrs=127.17.188.126:35593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:23.270116 26930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:23.270324 26930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:23.275925 26956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:23.275966 26954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:23.276196 26930 server_base.cc:1034] running on GCE node
W20260620 23:04:23.275944 26953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:23.276490 26930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:23.276719 26930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:23.277868 26930 hybrid_clock.cc:648] HybridClock initialized: now 1781996663277848 us; error 38 us; skew 500 ppm
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996663, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:23.280440 26930 init.cc:377] Logged in from keytab as oryx/127.17.188.65@KRBTEST.COM (short username oryx)
I20260620 23:04:23.280923 26930 webserver.cc:460] Webserver started at http://127.17.188.65:36001/ using document root <none> and password file <none>
I20260620 23:04:23.281183 26930 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:23.281250 26930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:23.282968 26930 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:23.283624 26963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.283876 26930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:23.283967 26930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "715665c6a92a4072868a3091c441ca92"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "b356903fc3d26ce6e2840e7f49ead2f1"
server_key_iv: "f04c6c350dce981437f60cb2cf3f4351"
server_key_version: "encryptionkey@0"
I20260620 23:04:23.284353 26930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:23.317824 26930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:23.320184 26930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:23.320365 26930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:23.320624 26930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:23.321010 26930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:23.321064 26930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.321117 26930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:23.321166 26930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.328634 26930 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:32937
I20260620 23:04:23.328666 27076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:32937 every 8 connection(s)
I20260620 23:04:23.329047 26930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:04:23.332597 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 26930
I20260620 23:04:23.332929 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:38021
--local_ip_for_outbound_sockets=127.17.188.66
--tserver_master_addrs=127.17.188.126:35593
--webserver_port=40165
--webserver_interface=127.17.188.66
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.188.66
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:23.338419 27079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.329731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:59953)
0620 23:04:23.329932 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:23.329977 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:23.330594 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:23.331020 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:23.331024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:23.331721 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:23.331971 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.331977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.333084 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.333085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.334431 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.334434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.335292 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.335294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.335390 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.335888 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:23.335902 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:23.338284 (+ 2382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":108,"threads_started":1}
W20260620 23:04:23.338647 27077 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:35593 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.188.126:35593: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
W20260620 23:04:23.338784 27080 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:23.329857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:59953 (local address 127.17.188.126:35593)
0620 23:04:23.330027 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.330040 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.330066 (+ 26us) server_negotiation.cc:407] Connection header received
0620 23:04:23.330770 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.330776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.330832 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.330948 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.332082 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.332966 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.334529 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.335194 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.335310 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.338461 (+ 3151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:59953: BlockingRecv error: recv got EOF from 127.17.188.65:59953 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:23.343499 27079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.338894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:34987)
0620 23:04:23.339060 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:23.339068 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:23.339141 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:23.339308 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:23.339309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:23.339442 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:23.339483 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.339484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.340116 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.340117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.340641 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.340643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.341019 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.341020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.341109 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.341580 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:23.341585 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:23.343424 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153}
W20260620 23:04:23.343549 27080 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:23.338932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:34987 (local address 127.17.188.126:35593)
0620 23:04:23.338993 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.338995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.339127 (+ 132us) server_negotiation.cc:407] Connection header received
0620 23:04:23.339225 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.339225 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.339239 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.339305 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.339558 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.340014 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.340746 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.340935 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.341018 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.343490 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:34987: BlockingRecv error: recv got EOF from 127.17.188.65:34987 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:23.348266 27079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.343725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:33427)
0620 23:04:23.343772 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:23.343783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:23.343832 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:23.344021 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:23.344021 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:23.344210 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:23.344238 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.344239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.344954 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.344955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.345620 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.345622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.346004 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.346005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.346064 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.346507 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:23.346511 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:23.348178 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 23:04:23.348381 27077 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:04:23.348318 27082 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:23.343770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:33427 (local address 127.17.188.126:35593)
0620 23:04:23.343907 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.343909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.343916 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:04:23.343935 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.343935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.343945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.344017 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.344353 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.344857 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.345703 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.345913 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.346003 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.348256 (+ 2253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:33427: BlockingRecv error: recv got EOF from 127.17.188.65:33427 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
W20260620 23:04:23.413090 27081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:23.413305 27081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:23.413367 27081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:23.414491 27081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:23.414551 27081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:23.414594 27081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:04:23.415992 27081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.66.keytab
--principal=oryx/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:38021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=40165
--tserver_master_addrs=127.17.188.126:35593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:23.416229 27081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:23.416435 27081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:23.421036 27088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:23.421175 27090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:23.421211 27087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:23.421190 27081 server_base.cc:1034] running on GCE node
I20260620 23:04:23.421590 27081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:23.421787 27081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:23.422917 27081 hybrid_clock.cc:648] HybridClock initialized: now 1781996663422906 us; error 27 us; skew 500 ppm
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996663, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:23.425297 27081 init.cc:377] Logged in from keytab as oryx/127.17.188.66@KRBTEST.COM (short username oryx)
I20260620 23:04:23.425746 27081 webserver.cc:460] Webserver started at http://127.17.188.66:40165/ using document root <none> and password file <none>
I20260620 23:04:23.425949 27081 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:23.426018 27081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:23.427183 27081 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:23.427729 27097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.427886 27081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:23.427966 27081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "4080bd858b7b44849457e69631135648"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "85a5355577cf27c62d7f242c8a3916a3"
server_key_iv: "5419d75bdc5ca5d574a9bf3d757ca547"
server_key_version: "encryptionkey@0"
I20260620 23:04:23.428243 27081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:23.444458 27081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:23.446624 27081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:23.446774 27081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:23.446969 27081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:23.447237 27081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:23.447268 27081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.447304 27081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:23.447341 27081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.453526 27081 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:38021
I20260620 23:04:23.453557 27210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:38021 every 8 connection(s)
I20260620 23:04:23.453922 27081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:04:23.457988 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 27081
I20260620 23:04:23.458309 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:41543
--local_ip_for_outbound_sockets=127.17.188.67
--tserver_master_addrs=127.17.188.126:35593
--webserver_port=42283
--webserver_interface=127.17.188.67
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.188.67
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:23.463060 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.454595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:38609)
0620 23:04:23.454748 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:23.454782 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:23.455280 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:23.455547 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:23.455551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:23.456122 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:23.456376 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.456382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.457065 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.457066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.458650 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.458653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.459227 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.459228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.459382 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.459993 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:23.460003 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:23.462826 (+ 2823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
W20260620 23:04:23.463136 27082 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:23.454593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:38609 (local address 127.17.188.126:35593)
0620 23:04:23.454658 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.454659 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.454863 (+ 204us) server_negotiation.cc:407] Connection header received
0620 23:04:23.455408 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.455410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.455425 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.455482 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.456467 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.456954 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.458755 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.459094 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.459162 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.463019 (+ 3857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:38609: BlockingRecv error: recv got EOF from 127.17.188.66:38609 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260620 23:04:23.464039 27211 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:35593 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.188.126:35593: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 23:04:23.470439 27082 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:23.464423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:36047 (local address 127.17.188.126:35593)
0620 23:04:23.464968 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.464970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.464997 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:04:23.465018 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.465019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.465030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.465087 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.465448 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.466151 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.467529 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.467774 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.467828 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.470389 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:36047: BlockingRecv error: recv got EOF from 127.17.188.66:36047 (error 108)
Metrics: {"server-negotiator.queue_time_us":522}
I20260620 23:04:23.470438 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.464333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:36047)
0620 23:04:23.464400 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:23.464414 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:23.464485 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:23.465112 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:23.465113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:23.465285 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:23.465371 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.465373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.466247 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.466248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.467437 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.467440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.467850 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.467851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.467935 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.468308 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:23.468314 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:23.470323 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:23.475287 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.470883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:38701)
0620 23:04:23.470990 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:23.471003 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:23.471062 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:23.471180 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:23.471181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:23.471332 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:23.471362 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.471364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.471894 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.471894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.472773 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.472775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.473078 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.473079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.473134 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.473508 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:23.473513 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:23.475223 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260620 23:04:23.475343 27082 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:23.470941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:38701 (local address 127.17.188.126:35593)
0620 23:04:23.470986 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.470987 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.471052 (+ 65us) server_negotiation.cc:407] Connection header received
0620 23:04:23.471122 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.471123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.471131 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.471172 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.471423 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.471826 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.472846 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.473013 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.473062 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.475292 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:38701: BlockingRecv error: recv got EOF from 127.17.188.66:38701 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260620 23:04:23.475394 27211 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:04:23.548913 27214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:23.549124 27214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:23.549151 27214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:23.550264 27214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:23.550300 27214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:23.550326 27214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:04:23.551612 27214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/krb5kdc/oryx_127.17.188.67.keytab
--principal=oryx/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:41543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=42283
--tserver_master_addrs=127.17.188.126:35593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:23.551775 27214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:23.551932 27214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:23.556811 27220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:23.556859 27222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:23.556916 27214 server_base.cc:1034] running on GCE node
W20260620 23:04:23.556797 27219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:23.557215 27214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:23.557497 27214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:23.558650 27214 hybrid_clock.cc:648] HybridClock initialized: now 1781996663558640 us; error 25 us; skew 500 ppm
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996663, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:23.565515 27214 init.cc:377] Logged in from keytab as oryx/127.17.188.67@KRBTEST.COM (short username oryx)
I20260620 23:04:23.565968 27214 webserver.cc:460] Webserver started at http://127.17.188.67:42283/ using document root <none> and password file <none>
I20260620 23:04:23.566185 27214 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:23.566262 27214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:23.567564 27214 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:04:23.568166 27229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.568306 27214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:04:23.568395 27214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "a0bd6dde23bd4c55b164e3c4108b7d0e"
format_stamp: "Formatted at 2026-06-20 23:04:22 on dist-test-slave-zpss"
server_key: "1514648ce192740af14170410e1eadc6"
server_key_iv: "a2d8f7fbb0f719301cb42b12f42d30b4"
server_key_version: "encryptionkey@0"
I20260620 23:04:23.568701 27214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:23.581861 27214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:23.583532 27214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:23.583634 27214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:23.583792 27214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:23.584013 27214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:23.584041 27214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.584061 27214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:23.584074 27214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.589987 27214 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:41543
I20260620 23:04:23.590034 27342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:41543 every 8 connection(s)
I20260620 23:04:23.590369 27214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:04:23.594488 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 27214
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 23:04:23.597592 27082 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:23.590972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:58165 (local address 127.17.188.126:35593)
0620 23:04:23.591037 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.591039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.591242 (+ 203us) server_negotiation.cc:407] Connection header received
0620 23:04:23.591740 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.591742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.591753 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.591811 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.592693 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.593187 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.594183 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.594438 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.594516 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.597536 (+ 3020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:58165: BlockingRecv error: recv got EOF from 127.17.188.67:58165 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:04:23.597656 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.590974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:58165)
0620 23:04:23.591129 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:23.591161 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:23.591637 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:23.591879 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:23.591882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:23.592419 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:23.592593 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.592597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.593286 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.593287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.594093 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.594095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.594521 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.594522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.594660 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.595231 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:23.595241 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:23.597471 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
W20260620 23:04:23.597851 27343 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:35593 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.188.126:35593: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:23.602454 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.598056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:47335)
0620 23:04:23.598087 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:23.598096 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:23.598132 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:23.598243 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:23.598244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:23.598367 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:23.598433 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.598434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.599298 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.599299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.599843 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.599846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.600356 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.600357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.600429 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.600775 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:23.600780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:23.602379 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":16}
W20260620 23:04:23.602504 27082 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:23.598105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:47335 (local address 127.17.188.126:35593)
0620 23:04:23.598147 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.598149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.598157 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:04:23.598180 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.598181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.598189 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.598232 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.598493 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.599220 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.599911 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.600225 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.600350 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.602441 (+ 2091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:47335: BlockingRecv error: recv got EOF from 127.17.188.67:47335 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:04:23.602686 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.595192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37312 (local address 127.17.188.126:35593)
0620 23:04:23.595238 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.595240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.595300 (+ 60us) server_negotiation.cc:407] Connection header received
0620 23:04:23.595385 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.595386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.595398 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.595454 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.596235 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.596861 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.597800 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.598094 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.598137 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.599495 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:23.599499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:23.599499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:23.599510 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:23.601194 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:23.601585 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:23.601588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:23.601589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:23.601709 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:23.601929 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:23.601930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:23.601930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:23.602186 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:23.602346 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:23.602457 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 23:04:23.602606 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:23 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:23.606722 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.603081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:45911)
0620 23:04:23.603126 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:23.603137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:23.603179 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:23.603324 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:23.603324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:23.603451 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:23.603474 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.603476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.604085 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.604086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.604555 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:23.604556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.604955 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.604955 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:23.604989 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.605282 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:23.605286 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:23.606677 (+ 1391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:04:23.606797 27080 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:23.603145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:45911 (local address 127.17.188.126:35593)
0620 23:04:23.603181 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.603183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.603192 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:23.603258 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.603259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.603269 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.603321 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.603540 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.603990 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.604617 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.604884 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.604946 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.606747 (+ 1801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:45911: BlockingRecv error: recv got EOF from 127.17.188.67:45911 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:04:23.606863 27343 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 23:04:24 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:24 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:24.353034 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:24.348801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:35891)
0620 23:04:24.348943 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:24.348952 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:24.348997 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:24.349115 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:24.349116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:24.349271 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:24.349356 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:24.349358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.349940 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.349940 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:24.350475 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:24.350476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.350778 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.350778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:24.350815 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:24.351218 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:24.351222 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:24.352937 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
W20260620 23:04:24.353078 27348 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:24.348880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:35891 (local address 127.17.188.126:35593)
0620 23:04:24.348993 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:24.348995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:24.349003 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:04:24.349041 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:24.349041 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:24.349052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:24.349098 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:24.349422 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.349880 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.350538 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.350716 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.350763 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:24.353007 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:35891: BlockingRecv error: recv got EOF from 127.17.188.65:35891 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":51,"threads_started":1}
Jun 20 23:04:24 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:24 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:24.480219 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:24.475690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:48203)
0620 23:04:24.475853 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:24.475865 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:24.475946 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:24.476097 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:24.476099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:24.476261 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:24.476303 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:24.476305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.476969 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.476971 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:24.477516 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:24.477517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.477888 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.477889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:24.477950 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:24.478438 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:24.478442 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:24.480116 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
W20260620 23:04:24.480254 27348 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:24.475767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:48203 (local address 127.17.188.126:35593)
0620 23:04:24.475841 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:24.475843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:24.475931 (+ 88us) server_negotiation.cc:407] Connection header received
0620 23:04:24.476029 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:24.476030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:24.476041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:24.476082 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:24.476381 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.476876 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.477612 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.477802 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.477888 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:24.480191 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:48203: BlockingRecv error: recv got EOF from 127.17.188.66:48203 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 20 23:04:24 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:24 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:24.612044 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:24.607391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:38275)
0620 23:04:24.607546 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:24.607558 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:24.607629 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:24.607803 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:24.607804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:24.607978 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:24.608031 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:24.608033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.608667 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.608668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:24.609189 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:24.609190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.609595 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.609596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:24.609655 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:24.610159 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:24.610164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:24.611945 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
W20260620 23:04:24.612080 27348 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:24.607475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:38275 (local address 127.17.188.126:35593)
0620 23:04:24.607550 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:24.607553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:24.607629 (+ 76us) server_negotiation.cc:407] Connection header received
0620 23:04:24.607683 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:24.607684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:24.607698 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:24.607798 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:24.608102 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.608585 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.609272 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:24.609515 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:24.609594 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:24.612015 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:38275: BlockingRecv error: recv got EOF from 127.17.188.67:38275 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 20 23:04:25 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:25 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:25.357719 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:25.353454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:47361)
0620 23:04:25.353588 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:25.353599 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:25.353646 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:25.353770 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:25.353771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:25.353943 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:25.353991 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:25.353993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.354610 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.354610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:25.355138 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:25.355139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.355462 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.355463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:25.355500 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:25.355982 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:25.355994 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:25.357667 (+ 1673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
W20260620 23:04:25.357816 27352 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:25.353524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:47361 (local address 127.17.188.126:35593)
0620 23:04:25.353654 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:25.353656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:25.353664 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:04:25.353694 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:25.353694 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:25.353704 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:25.353764 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:25.354053 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.354547 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.355205 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.355399 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.355452 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:25.357744 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:47361: BlockingRecv error: recv got EOF from 127.17.188.65:47361 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
Jun 20 23:04:25 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:25 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:25.485450 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:25.480716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:37665)
0620 23:04:25.480852 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:25.480863 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:25.480921 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:25.481044 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:25.481044 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:25.481212 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:25.481257 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:25.481259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.481945 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.481947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:25.482708 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:25.482710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.483051 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.483052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:25.483124 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:25.483627 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:25.483632 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:25.485359 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
W20260620 23:04:25.485493 27352 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:25.480851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:37665 (local address 127.17.188.126:35593)
0620 23:04:25.480904 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:25.480906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:25.480915 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:25.480979 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:25.480979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:25.480992 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:25.481040 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:25.481349 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.481849 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.482789 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.482982 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.483054 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:25.485433 (+ 2379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:37665: BlockingRecv error: recv got EOF from 127.17.188.66:37665 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 20 23:04:25 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:25 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:25.616946 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:25.612451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:47291)
0620 23:04:25.612586 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:25.612598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:25.612653 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:25.612827 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:25.612828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:25.612996 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:25.613039 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:25.613041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.613810 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.613811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:25.614349 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:25.614350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.614728 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.614728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:25.614786 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:25.615205 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:25.615209 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:25.616880 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
W20260620 23:04:25.617062 27352 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:25.612559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:47291 (local address 127.17.188.126:35593)
0620 23:04:25.612632 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:25.612634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:25.612643 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:25.612722 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:25.612723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:25.612753 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:25.612819 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:25.613119 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.613676 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.614447 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:25.614645 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:25.614726 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:25.616970 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:47291: BlockingRecv error: recv got EOF from 127.17.188.67:47291 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 20 23:04:26 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:26 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:26.362375 27355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:26.358207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:57743)
0620 23:04:26.358331 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:26.358341 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:26.358384 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:26.358553 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:26.358554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:26.358725 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:26.358771 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:26.358772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.359390 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.359390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:26.359891 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:26.359892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.360211 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.360211 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:26.360248 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:26.360656 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:26.360660 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:26.362319 (+ 1659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
W20260620 23:04:26.362464 27356 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:26.358295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:57743 (local address 127.17.188.126:35593)
0620 23:04:26.358428 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:26.358429 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:26.358437 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:04:26.358463 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:26.358464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:26.358475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:26.358535 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:26.358835 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.359329 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.359959 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.360154 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.360201 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:26.362399 (+ 2198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:57743: BlockingRecv error: recv got EOF from 127.17.188.65:57743 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
Jun 20 23:04:26 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:26 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:26.490190 27357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:26.485878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:41053)
0620 23:04:26.486013 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:26.486023 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:26.486074 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:26.486208 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:26.486209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:26.486356 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:26.486404 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:26.486405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.486996 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.486997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:26.487542 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:26.487543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.487968 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.487969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:26.488021 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:26.488463 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:26.488468 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:26.490135 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
W20260620 23:04:26.490264 27356 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:26.485954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:41053 (local address 127.17.188.126:35593)
0620 23:04:26.486017 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:26.486020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:26.486060 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:04:26.486129 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:26.486129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:26.486139 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:26.486194 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:26.486461 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.486935 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.487607 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.487882 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.487935 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:26.490221 (+ 2286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:41053: BlockingRecv error: recv got EOF from 127.17.188.66:41053 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 20 23:04:26 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:26 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:26.622805 27358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:26.617562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:50897)
0620 23:04:26.618211 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:26.618224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:26.618295 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:26.618569 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:26.618570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:26.618720 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:26.618768 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:26.618769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.619601 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.619602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:26.620074 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:26.620076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.620425 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.620425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:26.620491 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:26.620963 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:26.620969 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:26.622716 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":75,"threads_started":1}
W20260620 23:04:26.622836 27356 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:26.617936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:50897 (local address 127.17.188.126:35593)
0620 23:04:26.618455 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:26.618456 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:26.618462 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:04:26.618480 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:26.618481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:26.618491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:26.618550 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:26.618843 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.619522 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.620159 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:26.620356 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:26.620426 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:26.622781 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:50897: BlockingRecv error: recv got EOF from 127.17.188.67:50897 (error 108)
Metrics: {"server-negotiator.queue_time_us":193}
Jun 20 23:04:27 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:27 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:27.367105 27359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:27.362883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:58001)
0620 23:04:27.362988 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:27.362998 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:27.363043 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:27.363165 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:27.363166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:27.363309 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:27.363353 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:27.363354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.363962 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.363962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:27.364460 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:27.364461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.364761 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.364762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:27.364799 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:27.365324 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:27.365362 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:27.367052 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
W20260620 23:04:27.367251 27360 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:27.362907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:58001 (local address 127.17.188.126:35593)
0620 23:04:27.363016 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:27.363018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:27.363029 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:04:27.363097 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:27.363098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:27.363108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:27.363162 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:27.363407 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.363899 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.364522 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.364704 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.364750 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:27.367167 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:58001: BlockingRecv error: recv got EOF from 127.17.188.65:58001 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":59,"threads_started":1}
Jun 20 23:04:27 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:27 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:27.495195 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:27.490773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:41369)
0620 23:04:27.490945 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:27.490954 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:27.491035 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:27.491227 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:27.491228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:27.491431 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:27.491488 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:27.491490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.492099 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.492099 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:27.492587 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:27.492588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.492890 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.492891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:27.492926 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:27.493389 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:27.493394 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:27.495140 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":106,"threads_started":1}
W20260620 23:04:27.495301 27360 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:27.490854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:41369 (local address 127.17.188.126:35593)
0620 23:04:27.490921 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:27.490923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:27.491048 (+ 125us) server_negotiation.cc:407] Connection header received
0620 23:04:27.491097 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:27.491097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:27.491115 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:27.491186 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:27.491566 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.492037 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.492650 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.492831 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.492879 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:27.495229 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:41369: BlockingRecv error: recv got EOF from 127.17.188.66:41369 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 20 23:04:27 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:27 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:27.627646 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:27.623332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:47279)
0620 23:04:27.623476 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:27.623491 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:27.623558 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:27.623724 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:27.623736 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:27.623907 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:27.623949 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:27.623950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.624588 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.624589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:27.625137 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:27.625138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.625535 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.625536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:27.625597 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:27.625996 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:27.626000 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:27.627559 (+ 1559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
W20260620 23:04:27.627701 27360 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:27.623376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:47279 (local address 127.17.188.126:35593)
0620 23:04:27.623434 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:27.623436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:27.623541 (+ 105us) server_negotiation.cc:407] Connection header received
0620 23:04:27.623618 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:27.623619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:27.623657 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:27.623745 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:27.624032 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.624512 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.625215 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.625454 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.625533 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:27.627640 (+ 2107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:47279: BlockingRecv error: recv got EOF from 127.17.188.67:47279 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 20 23:04:28 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:28 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:28.372200 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:28.367515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:45573)
0620 23:04:28.367702 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:28.367716 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:28.367794 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:28.367996 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:28.367997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:28.368177 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:28.368224 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:28.368226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.368909 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.368909 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:28.369560 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:28.369562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.369878 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.369879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:28.369918 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:28.370340 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:28.370345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:28.372145 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
W20260620 23:04:28.372361 27364 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:28.367655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:45573 (local address 127.17.188.126:35593)
0620 23:04:28.367788 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:28.367790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:28.367798 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:04:28.367850 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:28.367850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:28.367877 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:28.367949 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:28.368303 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.368842 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.369629 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.369815 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.369867 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:28.372245 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:45573: BlockingRecv error: recv got EOF from 127.17.188.65:45573 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
Jun 20 23:04:28 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:28 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:28.500188 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:28.495619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:35507)
0620 23:04:28.495752 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:28.495764 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:28.495875 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:28.496003 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:28.496004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:28.496196 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:28.496242 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:28.496244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.496915 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.496916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:28.497486 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:28.497488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.497872 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.497873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:28.497927 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:28.498428 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:28.498434 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:28.500132 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
W20260620 23:04:28.500308 27364 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:28.495730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:35507 (local address 127.17.188.126:35593)
0620 23:04:28.495793 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:28.495795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:28.495862 (+ 67us) server_negotiation.cc:407] Connection header received
0620 23:04:28.495930 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:28.495931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:28.495943 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:28.495987 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:28.496336 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.496834 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.497572 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.497787 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.497872 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:28.500237 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:35507: BlockingRecv error: recv got EOF from 127.17.188.66:35507 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 20 23:04:28 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:28 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:28.635838 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:28.628144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:44703)
0620 23:04:28.628291 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:28.628301 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:28.628358 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:28.628482 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:28.628483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:28.628624 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:28.628668 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:28.628670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.629251 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.629252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:28.629764 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:28.629766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.630141 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.630142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:28.630197 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:28.633872 (+ 3675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:28.633878 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:28.635739 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
W20260620 23:04:28.635866 27364 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:28.628205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:44703 (local address 127.17.188.126:35593)
0620 23:04:28.628253 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:28.628255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:28.628379 (+ 124us) server_negotiation.cc:407] Connection header received
0620 23:04:28.628403 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:28.628404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:28.628416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:28.628469 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:28.628726 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.629192 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.629833 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:28.630009 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:28.630116 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:28.635811 (+ 5695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:44703: BlockingRecv error: recv got EOF from 127.17.188.67:44703 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 20 23:04:29 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:29 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:29.377072 27367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:29.372653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:58769)
0620 23:04:29.372789 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:29.372800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:29.372863 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:29.372993 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:29.372994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:29.373152 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:29.373201 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:29.373202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.373846 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.373847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:29.374371 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:29.374373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.374689 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.374690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:29.374728 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:29.375229 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:29.375233 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:29.376979 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
W20260620 23:04:29.377132 27368 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:29.372717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:58769 (local address 127.17.188.126:35593)
0620 23:04:29.372811 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:29.372813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:29.372858 (+ 45us) server_negotiation.cc:407] Connection header received
0620 23:04:29.372909 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:29.372910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:29.372920 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:29.372977 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:29.373258 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.373762 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.374439 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.374627 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.374677 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:29.377062 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:58769: BlockingRecv error: recv got EOF from 127.17.188.65:58769 (error 108)
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":45,"threads_started":1}
Jun 20 23:04:29 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:29 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:29.505136 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:29.500666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:55677)
0620 23:04:29.500790 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:29.500799 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:29.500846 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:29.500990 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:29.500990 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:29.501151 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:29.501199 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:29.501200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.501836 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.501836 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:29.502379 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:29.502381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.502685 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.502686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:29.502723 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:29.503144 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:29.503148 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:29.505063 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
W20260620 23:04:29.505240 27368 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:29.500725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:55677 (local address 127.17.188.126:35593)
0620 23:04:29.500812 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:29.500814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:29.500837 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:04:29.500906 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:29.500907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:29.500917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:29.500972 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:29.501257 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.501758 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.502446 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.502625 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.502676 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:29.505150 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:55677: BlockingRecv error: recv got EOF from 127.17.188.66:55677 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 20 23:04:29 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:29 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:29.640841 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:29.636345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:41943)
0620 23:04:29.636474 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:29.636488 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:29.636557 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:29.636685 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:29.636686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:29.636868 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:29.636912 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:29.636913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.637654 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.637656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:29.638183 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:29.638184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.638511 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.638512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:29.638568 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:29.639066 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:29.639071 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:29.640787 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
W20260620 23:04:29.641069 27368 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:29.636456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:41943 (local address 127.17.188.126:35593)
0620 23:04:29.636501 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:29.636503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:29.636539 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:04:29.636610 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:29.636611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:29.636624 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:29.636679 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:29.636991 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.637568 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.638266 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.638443 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.638512 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:29.641027 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:41943: BlockingRecv error: recv got EOF from 127.17.188.67:41943 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 20 23:04:30 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:30 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:30.382434 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:30.377619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:50173)
0620 23:04:30.377760 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:30.377783 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:30.377893 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:30.378226 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:30.378227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:30.378469 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:30.378537 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:30.378540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.379307 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.379308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:30.379826 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:30.379827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.380146 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.380146 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:30.380184 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:30.380591 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:30.380595 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:30.382367 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
W20260620 23:04:30.382524 27372 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:30.377701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:50173 (local address 127.17.188.126:35593)
0620 23:04:30.377832 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:30.377834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:30.377877 (+ 43us) server_negotiation.cc:407] Connection header received
0620 23:04:30.377952 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:30.377953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:30.377968 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:30.378164 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:30.378617 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.379249 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.379896 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.380084 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.380133 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:30.382452 (+ 2319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:50173: BlockingRecv error: recv got EOF from 127.17.188.65:50173 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
Jun 20 23:04:30 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:30 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:30.510491 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:30.505591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:32889)
0620 23:04:30.505710 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:30.505721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:30.505778 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:30.505907 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:30.505908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:30.506080 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:30.506128 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:30.506130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.507118 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.507119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:30.507636 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:30.507637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.507965 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.507966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:30.508093 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:30.508565 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:30.508570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:30.510396 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
W20260620 23:04:30.510537 27372 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:30.505685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:32889 (local address 127.17.188.126:35593)
0620 23:04:30.505754 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:30.505755 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:30.505775 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:04:30.505827 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:30.505828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:30.505838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:30.505895 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:30.506295 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.507040 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.507701 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.507880 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.507931 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:30.510475 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:32889: BlockingRecv error: recv got EOF from 127.17.188.66:32889 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 23:04:30 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:30 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:30.646031 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:30.641368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:54993)
0620 23:04:30.641513 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:30.641524 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:30.641589 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:30.641731 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:30.641732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:30.641894 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:30.641951 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:30.641952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.642576 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.642577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:30.643135 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:30.643136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.643445 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.643445 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:30.643485 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:30.643916 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:30.643921 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:30.645937 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
W20260620 23:04:30.646073 27372 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:30.641420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:54993 (local address 127.17.188.126:35593)
0620 23:04:30.641479 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:30.641481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:30.641598 (+ 117us) server_negotiation.cc:407] Connection header received
0620 23:04:30.641641 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:30.641641 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:30.641653 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:30.641703 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:30.642018 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.642487 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.643208 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:30.643382 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:30.643430 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:30.646011 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:54993: BlockingRecv error: recv got EOF from 127.17.188.67:54993 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 20 23:04:31 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:31 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:31.387308 27375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:31.382953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:55399)
0620 23:04:31.383127 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:31.383142 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:31.383212 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:31.383358 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:31.383359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:31.383531 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:31.383584 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:31.383585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.384223 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.384224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:31.384759 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:31.384761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.385080 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.385081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:31.385119 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:31.385577 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:31.385581 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:31.387254 (+ 1673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":66,"threads_started":1}
W20260620 23:04:31.387444 27376 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:31.383107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:55399 (local address 127.17.188.126:35593)
0620 23:04:31.383241 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:31.383242 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:31.383249 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:04:31.383270 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:31.383271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:31.383281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:31.383336 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:31.383650 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.384143 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.384834 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.385017 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.385064 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:31.387336 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:55399: BlockingRecv error: recv got EOF from 127.17.188.65:55399 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
Jun 20 23:04:31 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:31 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:31.515713 27377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:31.510947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:55229)
0620 23:04:31.511111 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:31.511121 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:31.511191 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:31.511348 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:31.511349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:31.511519 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:31.511564 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:31.511566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.512382 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.512383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:31.512906 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:31.512908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.513291 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.513292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:31.513390 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:31.513878 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:31.513884 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:31.515645 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
W20260620 23:04:31.515800 27376 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:31.511008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:55229 (local address 127.17.188.126:35593)
0620 23:04:31.511071 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:31.511073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:31.511178 (+ 105us) server_negotiation.cc:407] Connection header received
0620 23:04:31.511246 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:31.511247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:31.511272 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:31.511340 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:31.511638 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.512288 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.512986 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.513190 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.513290 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:31.515733 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:55229: BlockingRecv error: recv got EOF from 127.17.188.66:55229 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 20 23:04:31 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:31 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:31.650863 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:31.646556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:51747)
0620 23:04:31.646711 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:31.646722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:31.646798 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:31.646929 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:31.646930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:31.647096 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:31.647144 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:31.647146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.647792 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.647792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:31.648297 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:31.648299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.648619 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.648620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:31.648664 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:31.649144 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:31.649149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:31.650796 (+ 1647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
W20260620 23:04:31.650992 27376 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:31.646641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:51747 (local address 127.17.188.126:35593)
0620 23:04:31.646722 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:31.646728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:31.646795 (+ 67us) server_negotiation.cc:407] Connection header received
0620 23:04:31.646843 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:31.646843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:31.646855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:31.646907 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:31.647211 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.647710 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.648375 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:31.648553 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:31.648601 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:31.650906 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:51747: BlockingRecv error: recv got EOF from 127.17.188.67:51747 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 23:04:32 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:32 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:32.392292 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:32.387760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:49729)
0620 23:04:32.387924 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:32.387938 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:32.388004 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:32.388141 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:32.388142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:32.388301 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:32.388348 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:32.388350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.389012 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.389012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:32.389602 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:32.389604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.389922 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.389922 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:32.389960 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:32.390464 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:32.390469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:32.392230 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":79,"threads_started":1}
W20260620 23:04:32.392426 27380 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:32.387828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:49729 (local address 127.17.188.126:35593)
0620 23:04:32.387980 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:32.387982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:32.387990 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:04:32.388056 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:32.388056 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:32.388066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:32.388121 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:32.388407 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.388961 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.389665 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.389860 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.389914 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:32.392320 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:49729: BlockingRecv error: recv got EOF from 127.17.188.65:49729 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
Jun 20 23:04:32 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:32 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:32.520625 27381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:32.516229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:35649)
0620 23:04:32.516364 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:32.516379 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:32.516434 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:32.516540 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:32.516541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:32.516701 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:32.516747 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:32.516749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.517412 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.517413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:32.517932 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:32.517933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.518269 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.518270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:32.518321 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:32.518792 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:32.518797 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:32.520564 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
W20260620 23:04:32.520713 27380 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:32.516273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:35649 (local address 127.17.188.126:35593)
0620 23:04:32.516321 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:32.516323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:32.516422 (+ 99us) server_negotiation.cc:407] Connection header received
0620 23:04:32.516477 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:32.516478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:32.516487 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:32.516531 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:32.516806 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.517286 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.518003 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.518179 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.518227 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:32.520650 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:35649: BlockingRecv error: recv got EOF from 127.17.188.66:35649 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 20 23:04:32 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:32 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:32.656009 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:32.651451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:51347)
0620 23:04:32.651582 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:32.651591 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:32.651644 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:32.651763 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:32.651764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:32.651909 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:32.651952 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:32.651953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.652560 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.652561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:32.653049 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:32.653050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.653486 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.653487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:32.653526 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:32.653970 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:32.653974 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:32.655955 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
W20260620 23:04:32.656152 27380 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:32.651524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:51347 (local address 127.17.188.126:35593)
0620 23:04:32.651601 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:32.651602 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:32.651644 (+ 42us) server_negotiation.cc:407] Connection header received
0620 23:04:32.651684 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:32.651684 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:32.651695 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:32.651746 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:32.652011 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.652485 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.653130 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.653420 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.653478 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:32.656071 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:51347: BlockingRecv error: recv got EOF from 127.17.188.67:51347 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 20 23:04:33 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:33 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:33.397532 27383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:33.392703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:58853)
0620 23:04:33.392865 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:33.392882 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:33.392937 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:33.393112 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:33.393113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:33.393266 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:33.393355 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:33.393357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.394034 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.394035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:33.394634 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:33.394636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.395016 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.395017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:33.395095 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:33.395628 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:33.395633 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:33.397442 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":62,"threads_started":1}
W20260620 23:04:33.397598 27384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:33.392885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:58853 (local address 127.17.188.126:35593)
0620 23:04:33.392993 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:33.392995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:33.393002 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:04:33.393022 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:33.393022 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:33.393031 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:33.393093 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:33.393435 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.393948 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.394731 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.394932 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.395006 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:33.397530 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:58853: BlockingRecv error: recv got EOF from 127.17.188.65:58853 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":58,"threads_started":1}
Jun 20 23:04:33 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:33 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:33.525501 27385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:33.521253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:40799)
0620 23:04:33.521435 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:33.521447 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:33.521516 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:33.521683 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:33.521684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:33.521847 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:33.521893 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:33.521895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.522512 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.522513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:33.523026 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:33.523028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.523333 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.523334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:33.523371 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:33.523775 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:33.523779 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:33.525443 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":104,"threads_started":1}
W20260620 23:04:33.525599 27384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:33.521301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:40799 (local address 127.17.188.126:35593)
0620 23:04:33.521442 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:33.521444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:33.521543 (+ 99us) server_negotiation.cc:407] Connection header received
0620 23:04:33.521590 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:33.521591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:33.521604 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:33.521656 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:33.521956 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.522443 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.523093 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.523272 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.523323 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:33.525530 (+ 2207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:40799: BlockingRecv error: recv got EOF from 127.17.188.66:40799 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Jun 20 23:04:33 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:33 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:33.663797 27386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:33.656514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:42277)
0620 23:04:33.656634 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:33.656645 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:33.656710 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:33.656837 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:33.656838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:33.656992 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:33.657038 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:33.657039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.657711 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.657712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:33.658204 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:33.658206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.658561 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.658562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:33.658608 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:33.662022 (+ 3414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:33.662027 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:33.663741 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
W20260620 23:04:33.663921 27384 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:33.656611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:42277 (local address 127.17.188.126:35593)
0620 23:04:33.656655 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:33.656657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:33.656688 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:04:33.656762 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:33.656763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:33.656773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:33.656814 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:33.657101 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.657648 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.658292 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.658471 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.658535 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:33.663842 (+ 5307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:42277: BlockingRecv error: recv got EOF from 127.17.188.67:42277 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 20 23:04:34 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:34 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:34.402376 27387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:34.398030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:43875)
0620 23:04:34.398173 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:34.398183 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:34.398232 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:34.398372 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:34.398373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:34.398534 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:34.398579 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:34.398581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.399233 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.399234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:34.399777 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:34.399778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.400097 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.400098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:34.400135 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:34.400560 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:34.400564 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:34.402307 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
W20260620 23:04:34.402465 27388 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:34.398097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:43875 (local address 127.17.188.126:35593)
0620 23:04:34.398222 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:34.398223 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:34.398233 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:04:34.398283 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:34.398283 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:34.398293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:34.398352 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:34.398640 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.399132 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.399847 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.400035 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.400084 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:34.402396 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:43875: BlockingRecv error: recv got EOF from 127.17.188.65:43875 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
Jun 20 23:04:34 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:34 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:34.530370 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:34.525955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:40213)
0620 23:04:34.526081 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:34.526092 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:34.526157 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:34.526281 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:34.526282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:34.526461 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:34.526509 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:34.526510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.527135 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.527135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:34.527647 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:34.527649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.527949 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.527950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:34.527998 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:34.528492 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:34.528497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:34.530315 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
W20260620 23:04:34.530517 27388 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:34.526026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:40213 (local address 127.17.188.126:35593)
0620 23:04:34.526080 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:34.526082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:34.526160 (+ 78us) server_negotiation.cc:407] Connection header received
0620 23:04:34.526210 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:34.526211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:34.526222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:34.526268 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:34.526571 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.527071 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.527711 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.527891 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.527938 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:34.530424 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:40213: BlockingRecv error: recv got EOF from 127.17.188.66:40213 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 20 23:04:34 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:34 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:34.668944 27390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:34.664203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:41215)
0620 23:04:34.664327 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:34.664338 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:34.664388 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:34.664492 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:34.664492 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:34.664652 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:34.664695 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:34.664697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.665300 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.665301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:34.666166 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:34.666170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.666530 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.666531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:34.666597 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:34.667097 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:34.667101 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:34.668838 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
W20260620 23:04:34.668972 27388 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:34.664278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:41215 (local address 127.17.188.126:35593)
0620 23:04:34.664352 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:34.664354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:34.664378 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:04:34.664429 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:34.664430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:34.664441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:34.664488 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:34.664757 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.665226 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.666260 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:34.666461 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:34.666529 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:34.668908 (+ 2379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:41215: BlockingRecv error: recv got EOF from 127.17.188.67:41215 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 20 23:04:35 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:35 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:35.407657 27391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:35.402870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:41841)
0620 23:04:35.403004 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:35.403017 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:35.403065 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:35.403256 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:35.403257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:35.403438 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:35.403487 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:35.403502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.404337 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.404338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:35.404966 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:35.404968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.405303 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.405304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:35.405385 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:35.405837 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:35.405841 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:35.407590 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
W20260620 23:04:35.407725 27392 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:35.402950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:41841 (local address 127.17.188.126:35593)
0620 23:04:35.403095 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:35.403097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:35.403104 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:04:35.403123 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:35.403124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:35.403134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:35.403197 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:35.403585 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.404242 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.405042 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.405236 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.405288 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:35.407672 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:41841: BlockingRecv error: recv got EOF from 127.17.188.65:41841 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
Jun 20 23:04:35 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:35 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:35.535458 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:35.530816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:52147)
0620 23:04:35.530985 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:35.530997 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:35.531062 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:35.531170 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:35.531170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:35.531341 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:35.531386 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:35.531388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.532069 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.532070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:35.532861 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:35.532864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.533222 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.533223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:35.533279 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:35.533713 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:35.533717 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:35.535401 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
W20260620 23:04:35.535609 27392 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:35.530890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:52147 (local address 127.17.188.126:35593)
0620 23:04:35.530936 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:35.530937 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:35.531066 (+ 129us) server_negotiation.cc:407] Connection header received
0620 23:04:35.531105 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:35.531106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:35.531117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:35.531166 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:35.531463 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.531995 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.532928 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.533142 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.533213 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:35.535508 (+ 2295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:52147: BlockingRecv error: recv got EOF from 127.17.188.66:52147 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 20 23:04:35 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:35 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:35.674259 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:35.669466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:53485)
0620 23:04:35.669595 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:35.669605 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:35.669660 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:35.669783 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:35.669784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:35.669942 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:35.669986 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:35.669988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.670651 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.670652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:35.671590 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:35.671593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.671963 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.671964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:35.672006 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:35.672462 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:35.672467 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:35.674192 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
W20260620 23:04:35.674388 27392 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:35.669514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:53485 (local address 127.17.188.126:35593)
0620 23:04:35.669573 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:35.669578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:35.669657 (+ 79us) server_negotiation.cc:407] Connection header received
0620 23:04:35.669703 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:35.669704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:35.669715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:35.669768 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:35.670047 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.670528 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.671675 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.671897 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.671956 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:35.674281 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:53485: BlockingRecv error: recv got EOF from 127.17.188.67:53485 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 20 23:04:36 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:36 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:36.413429 27395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:36.408261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:39001)
0620 23:04:36.408381 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:36.408393 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:36.408461 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:36.408624 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:36.408625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:36.408788 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:36.408834 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:36.408836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.409841 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.409842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:36.410340 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:36.410341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.410815 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.410816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:36.410856 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:36.411343 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:36.411348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:36.413367 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
W20260620 23:04:36.413556 27396 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:36.408328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:39001 (local address 127.17.188.126:35593)
0620 23:04:36.408475 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:36.408478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:36.408487 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:36.408529 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:36.408530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:36.408542 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:36.408612 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:36.408903 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.409764 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.410417 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.410742 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.410808 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:36.413457 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:39001: BlockingRecv error: recv got EOF from 127.17.188.65:39001 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
Jun 20 23:04:36 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:36 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:36.540591 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:36.535881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:59787)
0620 23:04:36.536010 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:36.536020 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:36.536066 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:36.536236 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:36.536237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:36.536395 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:36.536443 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:36.536444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.537096 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.537097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:36.537691 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:36.537693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.538055 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.538056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:36.538095 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:36.538558 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:36.538562 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:36.540467 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
W20260620 23:04:36.540632 27396 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:36.536032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:59787 (local address 127.17.188.126:35593)
0620 23:04:36.536108 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:36.536110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:36.536118 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:04:36.536143 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:36.536144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:36.536155 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:36.536212 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:36.536511 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.537029 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.537768 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.537970 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.538048 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:36.540570 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:59787: BlockingRecv error: recv got EOF from 127.17.188.66:59787 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 20 23:04:36 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:36 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:36.678960 27398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:36.674684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:47703)
0620 23:04:36.674828 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:36.674838 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:36.674884 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:36.675016 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:36.675016 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:36.675177 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:36.675222 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:36.675223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.675909 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.675910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:36.676423 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:36.676425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.676735 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.676736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:36.676774 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:36.677220 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:36.677224 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:36.678896 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
W20260620 23:04:36.679106 27396 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:36.674802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:47703 (local address 127.17.188.126:35593)
0620 23:04:36.674873 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:36.674874 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:36.674884 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:04:36.674940 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:36.674941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:36.674952 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:36.674995 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:36.675309 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.675841 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.676492 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:36.676668 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:36.676718 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:36.679011 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:47703: BlockingRecv error: recv got EOF from 127.17.188.67:47703 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 20 23:04:37 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:37 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:37.418156 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:37.413822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:37855)
0620 23:04:37.413948 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:37.413958 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:37.414016 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:37.414145 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:37.414146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:37.414313 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:37.414358 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:37.414359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.414966 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.414967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:37.415471 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:37.415472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.415781 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.415781 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:37.415817 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:37.416292 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:37.416298 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:37.418101 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
W20260620 23:04:37.418267 27400 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:37.413894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:37855 (local address 127.17.188.126:35593)
0620 23:04:37.414009 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:37.414012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:37.414020 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:04:37.414061 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:37.414061 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:37.414071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:37.414128 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:37.414420 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.414904 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.415536 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.415722 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.415770 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:37.418189 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:37855: BlockingRecv error: recv got EOF from 127.17.188.65:37855 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
Jun 20 23:04:37 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:37 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:37.545696 27401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:37.541261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:53391)
0620 23:04:37.541431 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:37.541442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:37.541508 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:37.541627 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:37.541628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:37.541814 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:37.541859 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:37.541860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.542533 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.542534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:37.543061 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:37.543062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.543420 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.543421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:37.543476 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:37.543886 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:37.543890 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:37.545601 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
W20260620 23:04:37.545745 27400 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:37.541393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:53391 (local address 127.17.188.126:35593)
0620 23:04:37.541440 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:37.541442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:37.541490 (+ 48us) server_negotiation.cc:407] Connection header received
0620 23:04:37.541561 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:37.541561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:37.541574 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:37.541622 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:37.541949 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.542444 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.543143 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.543336 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.543418 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:37.545678 (+ 2260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:53391: BlockingRecv error: recv got EOF from 127.17.188.66:53391 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Jun 20 23:04:37 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:37 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.67@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:37.684079 27402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:37.679495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.67:55545)
0620 23:04:37.679635 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:37.679645 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:37.679694 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:37.679824 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:37.679825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:37.679977 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:37.680042 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:37.680043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.680840 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.680841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:37.681438 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:37.681440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.681808 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.681809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:37.681867 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:37.682309 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:37.682313 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:37.683989 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
W20260620 23:04:37.684118 27400 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:37.679603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:55545 (local address 127.17.188.126:35593)
0620 23:04:37.679668 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:37.679669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:37.679683 (+ 14us) server_negotiation.cc:407] Connection header received
0620 23:04:37.679748 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:37.679749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:37.679760 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:37.679806 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:37.680097 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.680735 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.681529 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:37.681728 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:37.681809 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:37.684058 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.67:55545: BlockingRecv error: recv got EOF from 127.17.188.67:55545 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 20 23:04:38 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:38 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.65@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:38.423107 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:38.418774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.65:46569)
0620 23:04:38.418908 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:38.418921 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:38.418975 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:38.419185 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:38.419186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:38.419361 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:38.419406 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:38.419408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.420033 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.420034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:38.420552 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:38.420554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.420853 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.420854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:38.420890 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:38.421297 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:38.421301 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:38.423043 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
W20260620 23:04:38.423246 27404 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:38.418860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:46569 (local address 127.17.188.126:35593)
0620 23:04:38.419057 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:38.419059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:38.419067 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:04:38.419089 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:38.419089 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:38.419102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:38.419166 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:38.419473 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.419954 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.420618 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.420797 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.420843 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:38.423153 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.65:46569: BlockingRecv error: recv got EOF from 127.17.188.65:46569 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":61,"threads_started":1}
Jun 20 23:04:38 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:04:38 dist-test-slave-zpss krb5kdc[26346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.188.66@KRBTEST.COM for oryx/127.17.188.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:04:38.550593 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:38.546190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35593 (local address 127.17.188.66:45199)
0620 23:04:38.546325 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:38.546335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:38.546391 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:38.546509 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:38.546509 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:38.546662 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:38.546710 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:38.546711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.547297 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.547298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:38.548116 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:38.548118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.548429 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.548430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:38.548466 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:38.548873 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:38.548877 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:38.550519 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.188.126:35593: Server oryx/127.17.188.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":55,"threads_started":1}
W20260620 23:04:38.550675 27404 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:38.546255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:45199 (local address 127.17.188.126:35593)
0620 23:04:38.546345 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:38.546346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:38.546376 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:04:38.546434 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:38.546435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:38.546445 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:38.546503 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:38.546772 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.547235 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.548187 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.548371 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.548422 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:38.550615 (+ 2193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.188.66:45199: BlockingRecv error: recv got EOF from 127.17.188.66:45199 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:04:38.600767 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 26930
I20260620 23:04:38.604955 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 27081
I20260620 23:04:38.609159 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 27214
I20260620 23:04:38.613049 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 26859
2026-06-20T23:04:38Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17385 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T23:04:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:38Z Disabled control of system clock
I20260620 23:04:38.640395 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42031
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 23:04:38.717438 27409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:38.717659 27409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:38.717691 27409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:38.718870 27409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:38.718932 27409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:38.718952 27409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:38.718971 27409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:38.719012 27409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:38.720269 27409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:38.720523 27409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:38.720798 27409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:38.725657 27417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:38.725780 27414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:38.725721 27415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:38.726114 27409 server_base.cc:1034] running on GCE node
I20260620 23:04:38.726332 27409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:38.726622 27409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:38.727790 27409 hybrid_clock.cc:648] HybridClock initialized: now 1781996678727779 us; error 25 us; skew 500 ppm
I20260620 23:04:38.731936 27409 webserver.cc:460] Webserver started at http://127.17.188.126:46599/ using document root <none> and password file <none>
I20260620 23:04:38.732188 27409 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:38.732239 27409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:38.732396 27409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:38.733490 27409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "78eea1f1d4a943aab613d775bf4ab150"
format_stamp: "Formatted at 2026-06-20 23:04:38 on dist-test-slave-zpss"
server_key: "5b63465fed85f70090f0985811965fc0"
server_key_iv: "30c6c2b44ab6e5361c4beece0a86e7c9"
server_key_version: "encryptionkey@0"
I20260620 23:04:38.733850 27409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "78eea1f1d4a943aab613d775bf4ab150"
format_stamp: "Formatted at 2026-06-20 23:04:38 on dist-test-slave-zpss"
server_key: "5b63465fed85f70090f0985811965fc0"
server_key_iv: "30c6c2b44ab6e5361c4beece0a86e7c9"
server_key_version: "encryptionkey@0"
I20260620 23:04:38.735101 27409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:04:38.735792 27423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:38.735987 27409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:04:38.736093 27409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "78eea1f1d4a943aab613d775bf4ab150"
format_stamp: "Formatted at 2026-06-20 23:04:38 on dist-test-slave-zpss"
server_key: "5b63465fed85f70090f0985811965fc0"
server_key_iv: "30c6c2b44ab6e5361c4beece0a86e7c9"
server_key_version: "encryptionkey@0"
I20260620 23:04:38.736166 27409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:38.758620 27409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:38.758872 27409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:38.759037 27409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:38.762720 27409 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:42031
I20260620 23:04:38.762751 27475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:42031 every 8 connection(s)
I20260620 23:04:38.763067 27409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:38.763665 27476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:38.764436 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 27409
I20260620 23:04:38.764523 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:04:38.764758 18161 external_mini_cluster.cc:1428] Setting key 71496c75c7afdd2abadab2723bbc75ea
I20260620 23:04:38.766687 27476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150: Bootstrap starting.
I20260620 23:04:38.767249 27476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:38.767562 27476 log.cc:824] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:38.768263 27476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150: No bootstrap required, opened a new log
I20260620 23:04:38.770092 27476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eea1f1d4a943aab613d775bf4ab150" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42031 } }
I20260620 23:04:38.770227 27476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:38.770248 27476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78eea1f1d4a943aab613d775bf4ab150, State: Initialized, Role: FOLLOWER
I20260620 23:04:38.770303 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:38.765272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33606 (local address 127.17.188.126:42031)
0620 23:04:38.765492 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:38.765499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:38.765521 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:04:38.765924 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:38.765930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:38.765982 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:38.766082 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:38.766343 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.767224 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.768216 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.768621 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.768711 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:38.769437 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:38.769440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:38.769441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:38.769597 (+ 156us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:38.769630 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:38.769640 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:38.769690 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:38.769790 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 23:04:38.769867 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":116,"threads_started":1}
I20260620 23:04:38.770392 27476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eea1f1d4a943aab613d775bf4ab150" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42031 } }
I20260620 23:04:38.770500 27476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:38.770534 27476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:38.770581 27476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:38.771041 27476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eea1f1d4a943aab613d775bf4ab150" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42031 } }
I20260620 23:04:38.771165 27476 leader_election.cc:302] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78eea1f1d4a943aab613d775bf4ab150; no voters:
I20260620 23:04:38.771354 27476 leader_election.cc:288] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:38.771427 27481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:38.771591 27481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [term 1 LEADER]: Becoming Leader. State: Replica: 78eea1f1d4a943aab613d775bf4ab150, State: Running, Role: LEADER
I20260620 23:04:38.771621 27476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:38.771680 27481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eea1f1d4a943aab613d775bf4ab150" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42031 } }
I20260620 23:04:38.772079 27482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78eea1f1d4a943aab613d775bf4ab150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eea1f1d4a943aab613d775bf4ab150" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42031 } } }
I20260620 23:04:38.772337 27482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:38.772258 27483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78eea1f1d4a943aab613d775bf4ab150. Latest consensus state: current_term: 1 leader_uuid: "78eea1f1d4a943aab613d775bf4ab150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78eea1f1d4a943aab613d775bf4ab150" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42031 } } }
I20260620 23:04:38.772526 27483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:38.772724 27491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:38.773150 27491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:38.774514 27491 catalog_manager.cc:1347] Generated new cluster ID: 94cc0ee3039745799cb8fccb972d990b
I20260620 23:04:38.774554 27491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:38.812630 27491 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:38.813174 27491 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:38.817271 27491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78eea1f1d4a943aab613d775bf4ab150: Generated new TSK 0
I20260620 23:04:38.817459 27491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:38.828529 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42031
--builtin_ntp_servers=127.17.188.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 23:04:38.906617 27500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:38.906796 27500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:38.906814 27500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:38.907826 27500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:38.907860 27500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:38.907881 27500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:04:38.909114 27500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:38.909322 27500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:38.909614 27500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:38.914198 27508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:38.914181 27505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:38.914208 27506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:38.914484 27500 server_base.cc:1034] running on GCE node
I20260620 23:04:38.914721 27500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:38.914986 27500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:38.916152 27500 hybrid_clock.cc:648] HybridClock initialized: now 1781996678916138 us; error 25 us; skew 500 ppm
I20260620 23:04:38.917686 27500 webserver.cc:460] Webserver started at http://127.17.188.65:44853/ using document root <none> and password file <none>
I20260620 23:04:38.917912 27500 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:38.917954 27500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:38.918094 27500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:38.919050 27500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "3414c363741c4273a9ddc183e18c7582"
format_stamp: "Formatted at 2026-06-20 23:04:38 on dist-test-slave-zpss"
server_key: "86dcde1cff444d643e2b99a82a0a3716"
server_key_iv: "ecc63ff7092da8be88db6e921a51d75c"
server_key_version: "encryptionkey@0"
I20260620 23:04:38.919363 27500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "3414c363741c4273a9ddc183e18c7582"
format_stamp: "Formatted at 2026-06-20 23:04:38 on dist-test-slave-zpss"
server_key: "86dcde1cff444d643e2b99a82a0a3716"
server_key_iv: "ecc63ff7092da8be88db6e921a51d75c"
server_key_version: "encryptionkey@0"
I20260620 23:04:38.920536 27500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:04:38.921172 27514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:38.921355 27500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:38.921458 27500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "3414c363741c4273a9ddc183e18c7582"
format_stamp: "Formatted at 2026-06-20 23:04:38 on dist-test-slave-zpss"
server_key: "86dcde1cff444d643e2b99a82a0a3716"
server_key_iv: "ecc63ff7092da8be88db6e921a51d75c"
server_key_version: "encryptionkey@0"
I20260620 23:04:38.921543 27500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:38.940855 27500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:38.941156 27500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:38.941308 27500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:38.941582 27500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:38.941888 27500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:38.941921 27500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:38.941972 27500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:38.941999 27500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:38.947353 27500 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:38797
I20260620 23:04:38.947429 27627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:38797 every 8 connection(s)
I20260620 23:04:38.947746 27500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:04:38.952754 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:38.948439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:46313 (local address 127.17.188.126:42031)
0620 23:04:38.948497 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:38.948499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:38.948758 (+ 259us) server_negotiation.cc:407] Connection header received
0620 23:04:38.949210 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:38.949211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:38.949221 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:38.949274 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:38.949953 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.950458 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.951421 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.951621 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.951689 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:38.952439 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:38.952440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:38.952441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:38.952445 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:38.952461 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:38.952467 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:38.952542 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:38.952640 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 23:04:38.952678 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:04:38.952872 27630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:38.948421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42031 (local address 127.17.188.65:46313)
0620 23:04:38.948599 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:38.948652 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:38.949104 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:38.949369 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:38.949373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:38.949385 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:38.949857 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:38.949861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.950547 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.950548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:38.951291 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:38.951292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.951708 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.951709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:38.951810 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:38.952297 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:04:38.952297 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:38.952323 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:04:38.952343 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:38.952548 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:38.952551 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:38.952554 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:04:38.952604 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 23:04:38.952675 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260620 23:04:38.952940 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 27500
I20260620 23:04:38.953023 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:04:38.953217 18161 external_mini_cluster.cc:1428] Setting key acf6f436d56e674e1401b38200201d3c
I20260620 23:04:38.953434 27628 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42031
I20260620 23:04:38.953639 27628 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:38.953873 27628 heartbeater.cc:511] Master 127.17.188.126:42031 requested a full tablet report, sending...
I20260620 23:04:38.954283 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42031
--builtin_ntp_servers=127.17.188.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 23:04:38.954444 27440 ts_manager.cc:194] Registered new tserver with Master: 3414c363741c4273a9ddc183e18c7582 (127.17.188.65:38797)
I20260620 23:04:38.955053 27440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.65:46313
W20260620 23:04:39.033497 27631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:39.033677 27631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:39.033696 27631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:39.034859 27631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:39.034897 27631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:39.034921 27631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:04:39.036247 27631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:39.036453 27631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:39.036715 27631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:04:39.041752 27631 server_base.cc:1034] running on GCE node
W20260620 23:04:39.041759 27637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.041698 27639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.041699 27636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:39.042065 27631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:39.042327 27631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:39.043496 27631 hybrid_clock.cc:648] HybridClock initialized: now 1781996679043477 us; error 32 us; skew 500 ppm
I20260620 23:04:39.044767 27631 webserver.cc:460] Webserver started at http://127.17.188.66:34453/ using document root <none> and password file <none>
I20260620 23:04:39.045027 27631 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:39.045100 27631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:39.045253 27631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:39.046291 27631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "6569b64cda5d478189256229ef5562e9"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "d85dd23acf0bd8ee3b9a5ee922ca86cb"
server_key_iv: "7b55b7e79029b02aff8d8b4cc84b4a1f"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.046641 27631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "6569b64cda5d478189256229ef5562e9"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "d85dd23acf0bd8ee3b9a5ee922ca86cb"
server_key_iv: "7b55b7e79029b02aff8d8b4cc84b4a1f"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.047838 27631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:04:39.048475 27645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.048628 27631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:39.048720 27631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "6569b64cda5d478189256229ef5562e9"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "d85dd23acf0bd8ee3b9a5ee922ca86cb"
server_key_iv: "7b55b7e79029b02aff8d8b4cc84b4a1f"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.048799 27631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:39.059742 27631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:39.059998 27631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:39.060124 27631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:39.060354 27631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:39.060649 27631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:39.060700 27631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.060737 27631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:39.060763 27631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.066159 27631 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:37065
I20260620 23:04:39.066229 27758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:37065 every 8 connection(s)
I20260620 23:04:39.066527 27631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:04:39.069267 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 27631
I20260620 23:04:39.069387 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:04:39.069576 18161 external_mini_cluster.cc:1428] Setting key f277f810e521f2c411b074c308e0ace1
I20260620 23:04:39.070508 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42031
--builtin_ntp_servers=127.17.188.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 23:04:39.071694 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.067191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:42257 (local address 127.17.188.126:42031)
0620 23:04:39.067279 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:39.067281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:39.067430 (+ 149us) server_negotiation.cc:407] Connection header received
0620 23:04:39.067850 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:39.067851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:39.067860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:39.067908 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:39.068490 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:39.068943 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:39.070214 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:39.070387 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:39.070476 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:39.071362 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:39.071363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:39.071364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:39.071368 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:39.071383 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:39.071387 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:39.071441 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:39.071578 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 23:04:39.071608 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 23:04:39.071902 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.067150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42031 (local address 127.17.188.66:42257)
0620 23:04:39.067313 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:39.067341 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:39.067774 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:39.067971 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:39.067974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:39.067982 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:39.068423 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:39.068426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:39.069036 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:39.069037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:39.070115 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:39.070117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:39.070491 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:39.070493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:39.070653 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:39.071208 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:04:39.071209 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:39.071246 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:04:39.071266 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:39.071443 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:39.071448 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:39.071451 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:04:39.071579 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 23:04:39.071685 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
I20260620 23:04:39.072301 27759 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42031
I20260620 23:04:39.072400 27759 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:39.072608 27759 heartbeater.cc:511] Master 127.17.188.126:42031 requested a full tablet report, sending...
I20260620 23:04:39.072942 27440 ts_manager.cc:194] Registered new tserver with Master: 6569b64cda5d478189256229ef5562e9 (127.17.188.66:37065)
I20260620 23:04:39.073361 27440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.66:42257
W20260620 23:04:39.148442 27762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:39.148646 27762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:39.148679 27762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:39.149842 27762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:39.149904 27762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:39.149945 27762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:04:39.151311 27762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:39.151546 27762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:39.151816 27762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:39.156723 27768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.156744 27770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.156705 27767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:39.156939 27762 server_base.cc:1034] running on GCE node
I20260620 23:04:39.157140 27762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:39.157387 27762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:39.158565 27762 hybrid_clock.cc:648] HybridClock initialized: now 1781996679158554 us; error 28 us; skew 500 ppm
I20260620 23:04:39.159852 27762 webserver.cc:460] Webserver started at http://127.17.188.67:46309/ using document root <none> and password file <none>
I20260620 23:04:39.160113 27762 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:39.160161 27762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:39.160326 27762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:39.161437 27762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "386cc1f7d4f94fe7a9215095d067d3a0"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "515be9ecb09b6725f57af2aded35fb9b"
server_key_iv: "0ffb11b568b0d395bfd94da09dc43087"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.161772 27762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "386cc1f7d4f94fe7a9215095d067d3a0"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "515be9ecb09b6725f57af2aded35fb9b"
server_key_iv: "0ffb11b568b0d395bfd94da09dc43087"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.163023 27762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:39.163666 27776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.163826 27762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.163923 27762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "386cc1f7d4f94fe7a9215095d067d3a0"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "515be9ecb09b6725f57af2aded35fb9b"
server_key_iv: "0ffb11b568b0d395bfd94da09dc43087"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.164004 27762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:39.183745 27762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:39.184075 27762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:39.184221 27762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:39.184427 27762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:39.184754 27762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:39.184808 27762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.184849 27762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:39.184876 27762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.190315 27762 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:41307
I20260620 23:04:39.190405 27889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:41307 every 8 connection(s)
I20260620 23:04:39.190722 27762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:04:39.195519 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 27762
I20260620 23:04:39.195608 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.191448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:54037 (local address 127.17.188.126:42031)
0620 23:04:39.191526 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:39.191528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:39.191705 (+ 177us) server_negotiation.cc:407] Connection header received
0620 23:04:39.192182 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:39.192183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:39.192194 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:39.192253 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:39.192870 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:39.193438 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:39.194359 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:39.194543 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:39.194599 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:39.195317 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:39.195319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:39.195320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:39.195325 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:39.195342 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:39.195346 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:39.195406 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:39.195486 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:04:39.195513 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 23:04:39.195621 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:04:39.195809 27892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.191407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42031 (local address 127.17.188.67:54037)
0620 23:04:39.191593 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:39.191624 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:39.192076 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:39.192318 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:39.192321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:39.192328 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:39.192796 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:39.192800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:39.193518 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:39.193518 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:39.194282 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:39.194284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:39.194618 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:39.194619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:39.194737 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:39.195182 (+ 445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:04:39.195182 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:39.195211 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:04:39.195231 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:39.195411 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:39.195415 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:39.195417 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:04:39.195467 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 23:04:39.195539 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":101,"threads_started":1}
I20260620 23:04:39.195824 18161 external_mini_cluster.cc:1428] Setting key 7b71c3c69ab14d0fdf50d887c71fd1b1
I20260620 23:04:39.196377 27890 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42031
I20260620 23:04:39.196466 27890 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:39.196661 27890 heartbeater.cc:511] Master 127.17.188.126:42031 requested a full tablet report, sending...
I20260620 23:04:39.197028 27439 ts_manager.cc:194] Registered new tserver with Master: 386cc1f7d4f94fe7a9215095d067d3a0 (127.17.188.67:41307)
I20260620 23:04:39.197472 27439 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.67:54037
I20260620 23:04:39.207535 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 23:04:39.209437 27479 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:39.208818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.17.188.126:42031)
0620 23:04:39.208890 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:39.208892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:39.209006 (+ 114us) server_negotiation.cc:407] Connection header received
0620 23:04:39.209091 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:39.209092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:39.209102 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:39.209196 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:39.209351 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33610: BlockingRecv error: recv got EOF from 127.0.0.1:33610 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 23:04:39.209507 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 27500
I20260620 23:04:39.214123 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 27631
I20260620 23:04:39.218055 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 27762
I20260620 23:04:39.222040 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 27409
2026-06-20T23:04:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (608 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T23:04:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:39Z Disabled control of system clock
I20260620 23:04:39.248261 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:33233
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:33233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 23:04:39.326486 27901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:39.326807 27901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:39.326834 27901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:39.327945 27901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:39.327984 27901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:39.327996 27901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:39.328008 27901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:39.328058 27901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:39.329463 27901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:33233
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:33233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:39.329713 27901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:39.329998 27901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:04:39.334630 27901 server_base.cc:1034] running on GCE node
W20260620 23:04:39.334702 27909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.334745 27907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.334652 27906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:39.335119 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:39.335433 27901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:39.336577 27901 hybrid_clock.cc:648] HybridClock initialized: now 1781996679336567 us; error 32 us; skew 500 ppm
I20260620 23:04:39.337842 27901 webserver.cc:460] Webserver started at http://127.17.188.126:37457/ using document root <none> and password file <none>
I20260620 23:04:39.338075 27901 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:39.338122 27901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:39.338284 27901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:39.339416 27901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "f91d2b772fda255fcc2d4f6cdae1aa88"
server_key_iv: "930427bece166b4bef04108b2ed7ffd6"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.340060 27901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "f91d2b772fda255fcc2d4f6cdae1aa88"
server_key_iv: "930427bece166b4bef04108b2ed7ffd6"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.341478 27901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:04:39.342161 27915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.342345 27901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:39.342430 27901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "f91d2b772fda255fcc2d4f6cdae1aa88"
server_key_iv: "930427bece166b4bef04108b2ed7ffd6"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.342516 27901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:39.345739 27901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:39.345985 27901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:39.346122 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:39.349774 27901 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:33233
I20260620 23:04:39.349839 27967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:33233 every 8 connection(s)
I20260620 23:04:39.350143 27901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:39.350739 27968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:39.351985 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 27901
I20260620 23:04:39.352100 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:04:39.352293 18161 external_mini_cluster.cc:1428] Setting key d337015d05f00f75e6076546f0cb80a2
I20260620 23:04:39.353873 27968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1: Bootstrap starting.
I20260620 23:04:39.354524 27968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:39.354846 27968 log.cc:824] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:39.354995 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.352779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.17.188.126:33233)
0620 23:04:39.352972 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:39.352981 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:39.353012 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:04:39.353573 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:39.353582 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:39.353636 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:39.353804 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:39.354402 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:39.354405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:39.354405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:39.354410 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:39.354455 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:39.354468 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:39.354568 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:39.354607 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 23:04:39.354786 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260620 23:04:39.355661 27968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1: No bootstrap required, opened a new log
I20260620 23:04:39.357035 27968 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33233 } }
I20260620 23:04:39.357192 27968 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:39.357216 27968 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ff1d8ab5c54559bbc87eba20fa2ef1, State: Initialized, Role: FOLLOWER
I20260620 23:04:39.357296 27968 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33233 } }
I20260620 23:04:39.357405 27968 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:39.357438 27968 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:39.357471 27968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:39.358146 27968 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33233 } }
I20260620 23:04:39.358232 27968 leader_election.cc:302] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5ff1d8ab5c54559bbc87eba20fa2ef1; no voters:
I20260620 23:04:39.358341 27968 leader_election.cc:288] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:39.358397 27973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:39.358547 27973 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [term 1 LEADER]: Becoming Leader. State: Replica: d5ff1d8ab5c54559bbc87eba20fa2ef1, State: Running, Role: LEADER
I20260620 23:04:39.358589 27968 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:39.358640 27973 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33233 } }
I20260620 23:04:39.359016 27975 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ff1d8ab5c54559bbc87eba20fa2ef1. Latest consensus state: current_term: 1 leader_uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33233 } } }
I20260620 23:04:39.359021 27974 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ff1d8ab5c54559bbc87eba20fa2ef1" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 33233 } } }
I20260620 23:04:39.359125 27975 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:39.359134 27974 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:39.359403 27980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:39.359853 27980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:39.361389 27980 catalog_manager.cc:1347] Generated new cluster ID: 22ef2f6be35f483da84c64e8a2fab7a9
I20260620 23:04:39.361442 27980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:39.396785 27980 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:39.396977 27980 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:39.404083 27980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5ff1d8ab5c54559bbc87eba20fa2ef1: Generated new TSK 0
I20260620 23:04:39.404273 27980 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:39.412374 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:33233
--builtin_ntp_servers=127.17.188.84:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 23:04:39.494390 27992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:39.494578 27992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:39.494599 27992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:39.495740 27992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:39.495779 27992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:39.495829 27992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:04:39.497124 27992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:33233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:39.497380 27992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:39.497659 27992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:04:39.502436 27992 server_base.cc:1034] running on GCE node
W20260620 23:04:39.502485 27998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.502583 27997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.502445 28000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:39.502919 27992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:39.503163 27992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:39.504426 27992 hybrid_clock.cc:648] HybridClock initialized: now 1781996679504407 us; error 34 us; skew 500 ppm
I20260620 23:04:39.505716 27992 webserver.cc:460] Webserver started at http://127.17.188.65:44483/ using document root <none> and password file <none>
I20260620 23:04:39.505935 27992 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:39.505980 27992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:39.506119 27992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:39.507094 27992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "2e64dd8e364c4394bf46a3de0db3d128"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "3f887efc657f88d20614b6885e0bd441"
server_key_iv: "440f2ba85480e84fd115a9f5ad20df9f"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.507436 27992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "2e64dd8e364c4394bf46a3de0db3d128"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "3f887efc657f88d20614b6885e0bd441"
server_key_iv: "440f2ba85480e84fd115a9f5ad20df9f"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.508658 27992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:04:39.509378 28006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.509553 27992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:39.509649 27992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "2e64dd8e364c4394bf46a3de0db3d128"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "3f887efc657f88d20614b6885e0bd441"
server_key_iv: "440f2ba85480e84fd115a9f5ad20df9f"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.509735 27992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:39.512840 27992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:39.513087 27992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:39.513217 27992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:39.513446 27992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:39.513758 27992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:39.513818 27992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.513855 27992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:39.513885 27992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.519136 27992 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:40947
I20260620 23:04:39.519207 28119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:40947 every 8 connection(s)
I20260620 23:04:39.519486 27992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:04:39.522179 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.520125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:60273 (local address 127.17.188.126:33233)
0620 23:04:39.520179 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:39.520180 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:39.520386 (+ 206us) server_negotiation.cc:407] Connection header received
0620 23:04:39.520756 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:39.520757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:39.520761 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:39.520811 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:39.521963 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:39.521965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:39.521965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:39.521969 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:39.521986 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:39.521990 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:39.522037 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:39.522100 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 23:04:39.522114 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 23:04:39.522364 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.520105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33233 (local address 127.17.188.65:60273)
0620 23:04:39.520267 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:39.520302 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:39.520677 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:39.520857 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:39.520860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:39.520867 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:39.521776 (+ 909us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:04:39.521776 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:39.521800 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:04:39.521853 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:39.522035 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:39.522040 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:39.522043 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:04:39.522112 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:04:39.522166 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260620 23:04:39.522921 28120 heartbeater.cc:347] Connected to a master server at 127.17.188.126:33233
I20260620 23:04:39.523023 28120 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:39.523162 28120 heartbeater.cc:511] Master 127.17.188.126:33233 requested a full tablet report, sending...
I20260620 23:04:39.523607 27932 ts_manager.cc:194] Registered new tserver with Master: 2e64dd8e364c4394bf46a3de0db3d128 (127.17.188.65:40947)
I20260620 23:04:39.527120 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 27992
I20260620 23:04:39.527221 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:04:39.527416 18161 external_mini_cluster.cc:1428] Setting key 15a254d64f55a2f82c3e9ca27421fe6b
I20260620 23:04:39.528280 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:33233
--builtin_ntp_servers=127.17.188.84:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 23:04:39.606909 28123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:39.607115 28123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:39.607137 28123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:39.608183 28123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:39.608214 28123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:39.608238 28123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:04:39.609491 28123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:33233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:39.609666 28123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:39.609905 28123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:39.614684 28128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.614712 28131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.614684 28129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:39.614962 28123 server_base.cc:1034] running on GCE node
I20260620 23:04:39.615161 28123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:39.615414 28123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:39.616557 28123 hybrid_clock.cc:648] HybridClock initialized: now 1781996679616545 us; error 28 us; skew 500 ppm
I20260620 23:04:39.617756 28123 webserver.cc:460] Webserver started at http://127.17.188.66:44169/ using document root <none> and password file <none>
I20260620 23:04:39.618001 28123 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:39.618043 28123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:39.618232 28123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:39.619277 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "a5934d4da0074564bf56f8332f84d035"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "cfbb3cbd8c741c85e06e3b0e61d0369d"
server_key_iv: "482f3867ce6ec368a2350632eaf4225a"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.619625 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "a5934d4da0074564bf56f8332f84d035"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "cfbb3cbd8c741c85e06e3b0e61d0369d"
server_key_iv: "482f3867ce6ec368a2350632eaf4225a"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.620837 28123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:04:39.621624 28137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.621801 28123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:39.621883 28123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "a5934d4da0074564bf56f8332f84d035"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "cfbb3cbd8c741c85e06e3b0e61d0369d"
server_key_iv: "482f3867ce6ec368a2350632eaf4225a"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.621953 28123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:39.625216 28123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:39.625476 28123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:39.625607 28123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:39.625797 28123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:39.626091 28123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:39.626121 28123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.626174 28123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:39.626226 28123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.631739 28123 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:39591
I20260620 23:04:39.631780 28250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:39591 every 8 connection(s)
I20260620 23:04:39.632048 28123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:04:39.632755 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 28123
I20260620 23:04:39.632848 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:04:39.633069 18161 external_mini_cluster.cc:1428] Setting key e5911697a65e36afca4411244bfa1cb7
I20260620 23:04:39.634052 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:33233
--builtin_ntp_servers=127.17.188.84:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 23:04:39.635214 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.632738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:53169 (local address 127.17.188.126:33233)
0620 23:04:39.632800 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:39.632802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:39.633151 (+ 349us) server_negotiation.cc:407] Connection header received
0620 23:04:39.633688 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:39.633689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:39.633695 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:39.633753 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:39.634946 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:39.634947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:39.634948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:39.634953 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:39.634968 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:39.634972 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:39.635041 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:39.635125 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 23:04:39.635140 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:04:39.635566 28253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.632725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33233 (local address 127.17.188.66:53169)
0620 23:04:39.633029 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:39.633064 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:39.633602 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:39.633826 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:39.633829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:39.633838 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:39.634802 (+ 964us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:04:39.634803 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:39.634828 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:04:39.634855 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:39.635054 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:39.635060 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:39.635062 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:04:39.635126 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:04:39.635185 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":92,"threads_started":1}
I20260620 23:04:39.636006 28251 heartbeater.cc:347] Connected to a master server at 127.17.188.126:33233
I20260620 23:04:39.636108 28251 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:39.636245 28251 heartbeater.cc:511] Master 127.17.188.126:33233 requested a full tablet report, sending...
I20260620 23:04:39.636611 27932 ts_manager.cc:194] Registered new tserver with Master: a5934d4da0074564bf56f8332f84d035 (127.17.188.66:39591)
W20260620 23:04:39.712793 28254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:39.712965 28254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:39.712986 28254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:39.714221 28254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:39.714260 28254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:39.714285 28254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:04:39.715502 28254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:33233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:39.715673 28254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:39.715898 28254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:39.720523 28259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:39.720520 28262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:39.720539 28254 server_base.cc:1034] running on GCE node
W20260620 23:04:39.720567 28260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:39.720959 28254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:39.721143 28254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:39.722271 28254 hybrid_clock.cc:648] HybridClock initialized: now 1781996679722242 us; error 46 us; skew 500 ppm
I20260620 23:04:39.723436 28254 webserver.cc:460] Webserver started at http://127.17.188.67:44089/ using document root <none> and password file <none>
I20260620 23:04:39.723652 28254 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:39.723696 28254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:39.723832 28254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:39.724865 28254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "f09ce57bf2694460a2be055b96c48d91"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "1e93aa0782c2b03624a058a677cd5837"
server_key_iv: "42800047f656a6309e9934d2d1f5869a"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.725211 28254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "f09ce57bf2694460a2be055b96c48d91"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "1e93aa0782c2b03624a058a677cd5837"
server_key_iv: "42800047f656a6309e9934d2d1f5869a"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.726462 28254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:04:39.727166 28268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.727366 28254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:04:39.727459 28254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "f09ce57bf2694460a2be055b96c48d91"
format_stamp: "Formatted at 2026-06-20 23:04:39 on dist-test-slave-zpss"
server_key: "1e93aa0782c2b03624a058a677cd5837"
server_key_iv: "42800047f656a6309e9934d2d1f5869a"
server_key_version: "encryptionkey@0"
I20260620 23:04:39.727545 28254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:39.730542 28254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:39.730782 28254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:39.730917 28254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:39.731123 28254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:39.731415 28254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:39.731463 28254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.731499 28254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:39.731524 28254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:39.737056 28254 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:35539
I20260620 23:04:39.737098 28381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:35539 every 8 connection(s)
I20260620 23:04:39.737443 28254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:04:39.739267 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 28254
I20260620 23:04:39.739359 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:04:39.739576 18161 external_mini_cluster.cc:1428] Setting key 34b9802da8e89a1c0e8a728c5de7721d
I20260620 23:04:39.740356 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.738088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:46969 (local address 127.17.188.126:33233)
0620 23:04:39.738125 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:39.738127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:39.738365 (+ 238us) server_negotiation.cc:407] Connection header received
0620 23:04:39.738843 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:39.738844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:39.738850 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:39.738917 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:39.740128 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:39.740130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:39.740130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:39.740134 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:39.740151 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:39.740155 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:39.740207 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:39.740279 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 23:04:39.740292 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:04:39.740549 28384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:39.738092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:33233 (local address 127.17.188.67:46969)
0620 23:04:39.738247 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:39.738274 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:39.738715 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:39.738962 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:39.738964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:39.738971 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:39.740000 (+ 1029us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:04:39.740001 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:39.740038 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:04:39.740064 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:39.740211 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:39.740216 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:39.740218 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:04:39.740291 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:04:39.740335 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260620 23:04:39.740973 28382 heartbeater.cc:347] Connected to a master server at 127.17.188.126:33233
I20260620 23:04:39.741053 28382 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:39.741146 28382 heartbeater.cc:511] Master 127.17.188.126:33233 requested a full tablet report, sending...
I20260620 23:04:39.741464 27932 ts_manager.cc:194] Registered new tserver with Master: f09ce57bf2694460a2be055b96c48d91 (127.17.188.67:35539)
I20260620 23:04:39.751516 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 23:04:39.753536 27971 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:39.752970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59100 (local address 127.17.188.126:33233)
0620 23:04:39.753038 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:39.753040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:39.753169 (+ 129us) server_negotiation.cc:407] Connection header received
0620 23:04:39.753192 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:39.753192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:39.753197 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:39.753251 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:39.753449 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59100: BlockingRecv error: recv got EOF from 127.0.0.1:59100 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:04:39.753552 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 27992
I20260620 23:04:39.758036 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 28123
I20260620 23:04:39.762466 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 28254
I20260620 23:04:39.766436 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 27901
2026-06-20T23:04:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (544 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:39 dist-test-slave-zpss krb5kdc[28392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:39 dist-test-slave-zpss krb5kdc[28392](info): set up 2 sockets
Jun 20 23:04:39 dist-test-slave-zpss krb5kdc[28392](info): commencing operation
krb5kdc: starting...
W20260620 23:04:40.793596 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:04:40 dist-test-slave-zpss krb5kdc[28392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:04:40.904320 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:44551
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:44551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:40.981722 28405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:40.981932 28405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:40.981985 28405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:40.983091 28405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:40.983152 28405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:40.983178 28405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:40.983207 28405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:40.983232 28405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:40.984625 28405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:44551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:44551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:40.984899 28405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:40.985146 28405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:04:40.989902 28405 server_base.cc:1034] running on GCE node
W20260620 23:04:40.989826 28411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:40.989821 28413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:40.989830 28410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:40.990239 28405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:40.990516 28405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:40.991660 28405 hybrid_clock.cc:648] HybridClock initialized: now 1781996680991651 us; error 28 us; skew 500 ppm
Jun 20 23:04:40 dist-test-slave-zpss krb5kdc[28392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996680, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:40.994097 28405 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:04:40.994495 28405 webserver.cc:460] Webserver started at http://127.17.188.126:44581/ using document root <none> and password file <none>
I20260620 23:04:40.994704 28405 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:40.994769 28405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:40.994918 28405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:40.995870 28405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "1c492278d9694f64a8e47fb92a7edd52"
format_stamp: "Formatted at 2026-06-20 23:04:40 on dist-test-slave-zpss"
server_key: "daec4ec3df5c785837a485f97bfb4116"
server_key_iv: "8b7b3761ee5907a069036668cba4cf77"
server_key_version: "encryptionkey@0"
I20260620 23:04:40.996284 28405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "1c492278d9694f64a8e47fb92a7edd52"
format_stamp: "Formatted at 2026-06-20 23:04:40 on dist-test-slave-zpss"
server_key: "daec4ec3df5c785837a485f97bfb4116"
server_key_iv: "8b7b3761ee5907a069036668cba4cf77"
server_key_version: "encryptionkey@0"
I20260620 23:04:40.997577 28405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:04:40.998299 28420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:40.998467 28405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:40.998562 28405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "1c492278d9694f64a8e47fb92a7edd52"
format_stamp: "Formatted at 2026-06-20 23:04:40 on dist-test-slave-zpss"
server_key: "daec4ec3df5c785837a485f97bfb4116"
server_key_iv: "8b7b3761ee5907a069036668cba4cf77"
server_key_version: "encryptionkey@0"
I20260620 23:04:40.998642 28405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:41.015705 28405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:41.017544 28405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:41.017710 28405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:41.021427 28405 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:44551
I20260620 23:04:41.021479 28472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:44551 every 8 connection(s)
I20260620 23:04:41.021824 28405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:41.022847 28473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:41.025044 28473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52: Bootstrap starting.
I20260620 23:04:41.025624 28473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:41.025851 28473 log.cc:824] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:41.026463 28473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52: No bootstrap required, opened a new log
I20260620 23:04:41.027885 28473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c492278d9694f64a8e47fb92a7edd52" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44551 } }
I20260620 23:04:41.028007 28473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:41.028028 28473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c492278d9694f64a8e47fb92a7edd52, State: Initialized, Role: FOLLOWER
I20260620 23:04:41.028184 28473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c492278d9694f64a8e47fb92a7edd52" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44551 } }
I20260620 23:04:41.028266 28473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:41.028288 28473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:41.028316 28473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:41.028769 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 28405
I20260620 23:04:41.028882 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:04:41.029065 18161 external_mini_cluster.cc:1428] Setting key f0c664e9f57652721d8eafd351d16b3c
I20260620 23:04:41.029089 28473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c492278d9694f64a8e47fb92a7edd52" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44551 } }
I20260620 23:04:41.029229 28473 leader_election.cc:302] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c492278d9694f64a8e47fb92a7edd52; no voters:
I20260620 23:04:41.029408 28473 leader_election.cc:288] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:41.029527 28476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:41.029667 28473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:41.029707 28476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [term 1 LEADER]: Becoming Leader. State: Replica: 1c492278d9694f64a8e47fb92a7edd52, State: Running, Role: LEADER
I20260620 23:04:41.029881 28476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c492278d9694f64a8e47fb92a7edd52" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44551 } }
I20260620 23:04:41.030402 28477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c492278d9694f64a8e47fb92a7edd52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c492278d9694f64a8e47fb92a7edd52" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44551 } } }
I20260620 23:04:41.030421 28480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c492278d9694f64a8e47fb92a7edd52. Latest consensus state: current_term: 1 leader_uuid: "1c492278d9694f64a8e47fb92a7edd52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c492278d9694f64a8e47fb92a7edd52" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44551 } } }
I20260620 23:04:41.030623 28477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:41.030638 28480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:41.031029 28490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:41.031683 28490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:41.033152 28490 catalog_manager.cc:1347] Generated new cluster ID: 5dbb8cb5f2d8439686470f880733bdb6
I20260620 23:04:41.033205 28490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:04:41.040808 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.029542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47592 (local address 127.17.188.126:44551)
0620 23:04:41.029727 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.029736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.030028 (+ 292us) server_negotiation.cc:407] Connection header received
0620 23:04:41.030190 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.030197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.030250 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.030401 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:41.031304 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.032528 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.033668 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.034277 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.034376 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.037101 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:41.037105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:41.037105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:41.037115 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:41.039194 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.039571 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.039574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.039574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.039690 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.039919 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.039920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.039920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.040216 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:41.040387 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.040456 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.040596 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260620 23:04:41.059862 28490 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:41.060406 28490 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:41.073766 28490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52: Generated new TSK 0
I20260620 23:04:41.073947 28490 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:04:41.097831 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44551
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:41.176970 28499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:41.177153 28499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:41.177175 28499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:41.178349 28499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:41.178390 28499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:41.178416 28499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:04:41.179734 28499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:41.179901 28499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:41.180076 28499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:41.184913 28507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:41.184914 28505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:41.185151 28504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:41.185195 28499 server_base.cc:1034] running on GCE node
I20260620 23:04:41.185464 28499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:41.185752 28499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:41.186901 28499 hybrid_clock.cc:648] HybridClock initialized: now 1781996681186890 us; error 27 us; skew 500 ppm
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:41.189488 28499 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:04:41.189936 28499 webserver.cc:460] Webserver started at http://127.17.188.65:42225/ using document root <none> and password file <none>
I20260620 23:04:41.190178 28499 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:41.190255 28499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:41.190387 28499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:41.191332 28499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "f9250e4aa9b94bff9e2f46cd397077ba"
format_stamp: "Formatted at 2026-06-20 23:04:41 on dist-test-slave-zpss"
server_key: "2584d721b2426b3fd1d5312bb410bb9c"
server_key_iv: "7414c3bd220c2d4fb857fc89bfdbe33e"
server_key_version: "encryptionkey@0"
I20260620 23:04:41.191707 28499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "f9250e4aa9b94bff9e2f46cd397077ba"
format_stamp: "Formatted at 2026-06-20 23:04:41 on dist-test-slave-zpss"
server_key: "2584d721b2426b3fd1d5312bb410bb9c"
server_key_iv: "7414c3bd220c2d4fb857fc89bfdbe33e"
server_key_version: "encryptionkey@0"
I20260620 23:04:41.193024 28499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:04:41.193881 28514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:41.194077 28499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:04:41.194187 28499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "f9250e4aa9b94bff9e2f46cd397077ba"
format_stamp: "Formatted at 2026-06-20 23:04:41 on dist-test-slave-zpss"
server_key: "2584d721b2426b3fd1d5312bb410bb9c"
server_key_iv: "7414c3bd220c2d4fb857fc89bfdbe33e"
server_key_version: "encryptionkey@0"
I20260620 23:04:41.194270 28499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:41.207983 28499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:41.210098 28499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:41.210242 28499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:41.210448 28499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:41.210745 28499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:41.210796 28499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:41.210834 28499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:41.210860 28499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:41.216634 28499 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:42439
I20260620 23:04:41.216681 28627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:42439 every 8 connection(s)
I20260620 23:04:41.216943 28499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:04:41.222021 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 28499
I20260620 23:04:41.222097 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:04:41.222289 18161 external_mini_cluster.cc:1428] Setting key 0faefd0b98684115fbff1b019e3a91b6
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:04:41.225692 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.217585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:33973 (local address 127.17.188.126:44551)
0620 23:04:41.217656 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.217659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.217865 (+ 206us) server_negotiation.cc:407] Connection header received
0620 23:04:41.218302 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.218303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.218312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.218363 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:41.219197 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.219661 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.220549 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.220719 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.220765 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.222888 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:41.222889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:41.222890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:41.222897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:41.224242 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.224654 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.224654 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.224655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.224728 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.225001 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.225001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.225002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.225110 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:41.225184 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.225518 (+ 334us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.225602 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:04:41.225855 28630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.217576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:44551 (local address 127.17.188.65:33973)
0620 23:04:41.217754 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:41.217785 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:41.218217 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:41.218413 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:41.218416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:41.218950 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:41.219126 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.219129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.219736 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.219737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.220482 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.220483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.220782 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.220783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.220866 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.221376 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:41.221383 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:41.222804 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:41.224345 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:41.224352 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:41.224354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:41.224551 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:41.224797 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:41.224798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:41.224798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:41.224929 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:41.225175 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:41.225181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:41.225312 (+ 131us) client_negotiation.cc:769] Sending connection context
0620 23:04:41.225494 (+ 182us) client_negotiation.cc:240] Negotiation successful
0620 23:04:41.225637 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260620 23:04:41.226321 28628 heartbeater.cc:347] Connected to a master server at 127.17.188.126:44551
I20260620 23:04:41.226399 28628 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:41.226562 28628 heartbeater.cc:511] Master 127.17.188.126:44551 requested a full tablet report, sending...
I20260620 23:04:41.227181 28437 ts_manager.cc:194] Registered new tserver with Master: f9250e4aa9b94bff9e2f46cd397077ba (127.17.188.65:42439)
I20260620 23:04:41.227756 28437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:33973
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:04:41.245033 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44551
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:41.323570 28633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:41.323796 28633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:41.323832 28633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:41.324959 28633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:41.325030 28633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:41.325074 28633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:04:41.326546 28633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:41.326809 28633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:41.327033 28633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:41.331911 28638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:41.331945 28641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:41.331918 28639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:41.332219 28633 server_base.cc:1034] running on GCE node
I20260620 23:04:41.332406 28633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:41.332664 28633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:41.333844 28633 hybrid_clock.cc:648] HybridClock initialized: now 1781996681333835 us; error 27 us; skew 500 ppm
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:41.336169 28633 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:04:41.336601 28633 webserver.cc:460] Webserver started at http://127.17.188.66:45643/ using document root <none> and password file <none>
I20260620 23:04:41.336827 28633 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:41.336889 28633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:41.337030 28633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:41.338045 28633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "bdf87b3d10e24554ae27f55b007c3570"
format_stamp: "Formatted at 2026-06-20 23:04:41 on dist-test-slave-zpss"
server_key: "e95d5b2c2dfaca4251ebe4953e2ba3f8"
server_key_iv: "3a719411538ad98d7becd1e4db795aa7"
server_key_version: "encryptionkey@0"
I20260620 23:04:41.338408 28633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "bdf87b3d10e24554ae27f55b007c3570"
format_stamp: "Formatted at 2026-06-20 23:04:41 on dist-test-slave-zpss"
server_key: "e95d5b2c2dfaca4251ebe4953e2ba3f8"
server_key_iv: "3a719411538ad98d7becd1e4db795aa7"
server_key_version: "encryptionkey@0"
I20260620 23:04:41.339658 28633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:04:41.340364 28648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:41.340529 28633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:04:41.340623 28633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "bdf87b3d10e24554ae27f55b007c3570"
format_stamp: "Formatted at 2026-06-20 23:04:41 on dist-test-slave-zpss"
server_key: "e95d5b2c2dfaca4251ebe4953e2ba3f8"
server_key_iv: "3a719411538ad98d7becd1e4db795aa7"
server_key_version: "encryptionkey@0"
I20260620 23:04:41.340700 28633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:41.370000 28633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:41.371735 28633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:41.371889 28633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:41.372104 28633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:41.372385 28633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:41.372437 28633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:41.372467 28633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:41.372485 28633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:41.377991 28633 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:42947
I20260620 23:04:41.378041 28761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:42947 every 8 connection(s)
I20260620 23:04:41.378312 28633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:04:41.379279 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 28633
I20260620 23:04:41.379381 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:04:41.379572 18161 external_mini_cluster.cc:1428] Setting key c377710607d0e0687bc1cebf140189d2
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
I20260620 23:04:41.388551 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.378884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:38235 (local address 127.17.188.126:44551)
0620 23:04:41.378939 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.378941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.379191 (+ 250us) server_negotiation.cc:407] Connection header received
0620 23:04:41.379822 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.379824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.379835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.379902 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:41.380770 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.381297 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.382807 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.383009 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.383069 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.385762 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:41.385763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:41.385763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:41.385769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:41.387184 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.387667 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.387668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.387668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.387715 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.387975 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.387976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.387976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.388091 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:41.388154 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.388406 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.388487 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:04:41.388691 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.378846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:44551 (local address 127.17.188.66:38235)
0620 23:04:41.379049 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:41.379105 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:41.379706 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:41.379965 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:41.379969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:41.380495 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:41.380674 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.380678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.381428 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.381429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.382716 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.382718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.383100 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.383101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.383243 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.383734 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:41.383744 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:41.385670 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:41.387268 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:41.387274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:41.387276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:41.387583 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:41.387777 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:41.387778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:41.387779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:41.387912 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:41.388179 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:41.388184 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:41.388287 (+ 103us) client_negotiation.cc:769] Sending connection context
0620 23:04:41.388364 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 23:04:41.388496 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":114,"threads_started":1}
I20260620 23:04:41.389163 28762 heartbeater.cc:347] Connected to a master server at 127.17.188.126:44551
I20260620 23:04:41.389258 28762 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:41.389431 28762 heartbeater.cc:511] Master 127.17.188.126:44551 requested a full tablet report, sending...
I20260620 23:04:41.389832 28437 ts_manager.cc:194] Registered new tserver with Master: bdf87b3d10e24554ae27f55b007c3570 (127.17.188.66:42947)
I20260620 23:04:41.390218 28437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:38235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:04:41.404137 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44551
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:41.484368 28767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:41.484548 28767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:41.484570 28767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:41.485754 28767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:41.485792 28767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:41.485818 28767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:04:41.487205 28767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:04:41.487416 28767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:41.487637 28767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:41.492390 28775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:41.492506 28772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:41.492400 28773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:41.492697 28767 server_base.cc:1034] running on GCE node
I20260620 23:04:41.492862 28767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:41.493122 28767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:41.494297 28767 hybrid_clock.cc:648] HybridClock initialized: now 1781996681494288 us; error 42 us; skew 500 ppm
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:41.496593 28767 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:04:41.497102 28767 webserver.cc:460] Webserver started at http://127.17.188.67:45207/ using document root <none> and password file <none>
I20260620 23:04:41.497282 28767 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:41.497366 28767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:41.497478 28767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:41.498456 28767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "1b61e83593b94841a11bf413fbeceb3f"
format_stamp: "Formatted at 2026-06-20 23:04:41 on dist-test-slave-zpss"
server_key: "1b08c05ef8cc681d46fc82394f267f59"
server_key_iv: "919c87624734095c1d63d8ece7cdb9e1"
server_key_version: "encryptionkey@0"
I20260620 23:04:41.498752 28767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "1b61e83593b94841a11bf413fbeceb3f"
format_stamp: "Formatted at 2026-06-20 23:04:41 on dist-test-slave-zpss"
server_key: "1b08c05ef8cc681d46fc82394f267f59"
server_key_iv: "919c87624734095c1d63d8ece7cdb9e1"
server_key_version: "encryptionkey@0"
I20260620 23:04:41.499965 28767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:04:41.500618 28782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:41.500820 28767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:04:41.500912 28767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "1b61e83593b94841a11bf413fbeceb3f"
format_stamp: "Formatted at 2026-06-20 23:04:41 on dist-test-slave-zpss"
server_key: "1b08c05ef8cc681d46fc82394f267f59"
server_key_iv: "919c87624734095c1d63d8ece7cdb9e1"
server_key_version: "encryptionkey@0"
I20260620 23:04:41.500988 28767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:41.511320 28767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:41.513028 28767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:41.513177 28767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:41.513435 28767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:41.513744 28767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:41.513777 28767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:41.513797 28767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:41.513844 28767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:41.519649 28767 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:40501
I20260620 23:04:41.519706 28895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:40501 every 8 connection(s)
I20260620 23:04:41.520026 28767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:04:41.528393 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 28767
I20260620 23:04:41.528518 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:04:41.528736 18161 external_mini_cluster.cc:1428] Setting key 3122ea74d2e642376cd6a813650c5573
I20260620 23:04:41.528939 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.520759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:41761 (local address 127.17.188.126:44551)
0620 23:04:41.520846 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.520848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.520991 (+ 143us) server_negotiation.cc:407] Connection header received
0620 23:04:41.521589 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.521590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.521602 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.521694 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:41.522581 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.523080 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.524058 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.524280 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.524357 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.526313 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:41.526314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:41.526315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:41.526322 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:41.527483 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.527876 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.527877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.527877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.527938 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.528181 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.528182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.528182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.528320 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:41.528396 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.528743 (+ 347us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.528836 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 23:04:41.529106 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.520708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:44551 (local address 127.17.188.67:41761)
0620 23:04:41.520866 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:41.520898 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:41.521419 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:41.521768 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:41.521772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:41.522310 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:41.522481 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.522485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.523180 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.523181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.523959 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.523961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.524362 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.524363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.524488 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.524900 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:41.524908 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:41.526205 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:41.527581 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:41.527588 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:41.527590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:41.527799 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:41.528006 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:41.528007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:41.528007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:41.528107 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:41.528409 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:41.528415 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:41.528582 (+ 167us) client_negotiation.cc:769] Sending connection context
0620 23:04:41.528705 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 23:04:41.528876 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260620 23:04:41.529675 28896 heartbeater.cc:347] Connected to a master server at 127.17.188.126:44551
I20260620 23:04:41.529783 28896 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:41.529996 28896 heartbeater.cc:511] Master 127.17.188.126:44551 requested a full tablet report, sending...
I20260620 23:04:41.530331 28437 ts_manager.cc:194] Registered new tserver with Master: 1b61e83593b94841a11bf413fbeceb3f (127.17.188.67:40501)
I20260620 23:04:41.530740 28437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:41761
I20260620 23:04:41.540594 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:04:41.547515 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.541996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.17.188.126:44551)
0620 23:04:41.542050 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.542053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.542140 (+ 87us) server_negotiation.cc:407] Connection header received
0620 23:04:41.542196 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.542197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.542204 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.542247 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:41.542741 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.543187 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.543913 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.544087 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.544167 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.545244 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:41.545245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:41.545245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:41.545252 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:41.546399 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.546752 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.546753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.546753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.546804 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:41.547006 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:41.547007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:41.547008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:41.547141 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:41.547246 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.547348 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.547419 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:04:41.549034 28437 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:04:41.550577 28437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:04:41.556547 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.552505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:42947 (local address 127.0.0.1:43288)
0620 23:04:41.552805 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:41.552846 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:41.552910 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:41.553134 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:41.553137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:41.553139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:41.553207 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.553209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.554537 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.554538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.555584 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.555586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.556222 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.556222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.556310 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.556323 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 23:04:41.556392 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:04:41.556455 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"spinlock_wait_cycles":896,"thread_start_us":61,"threads_started":1}
I20260620 23:04:41.556679 28909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.552828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33330 (local address 127.17.188.65:42439)
0620 23:04:41.553014 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.553018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.553027 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:41.553065 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.553067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.553078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.553167 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:41.553366 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.554160 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.555578 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.556311 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.556386 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.556444 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.556541 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.556590 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
I20260620 23:04:41.556710 28904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.552505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:40501 (local address 127.0.0.1:47528)
0620 23:04:41.552807 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:41.552846 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:41.552909 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:41.553505 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:41.553506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:41.553508 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:41.553541 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.553542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.554716 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.554717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.555614 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.555616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.556479 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.556480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.556543 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.556547 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:04:41.556594 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 23:04:41.556635 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":1,"thread_start_us":49,"threads_started":1}
I20260620 23:04:41.556803 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.552605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.17.188.66:42947)
0620 23:04:41.552788 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.552793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.552984 (+ 191us) server_negotiation.cc:407] Connection header received
0620 23:04:41.553023 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.553025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.553046 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.553132 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:41.553290 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.554433 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.555651 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.556133 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.556221 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.556279 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.556671 (+ 392us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.556721 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
I20260620 23:04:41.557001 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.552588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.17.188.67:40501)
0620 23:04:41.553284 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.553287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.553296 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:04:41.553327 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.553329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.553367 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.553437 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:41.553604 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.554618 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.555702 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.556347 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.556403 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.556444 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.556853 (+ 409us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.556921 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":94,"threads_started":1}
I20260620 23:04:41.557176 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.552718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:42439 (local address 127.0.0.1:33330)
0620 23:04:41.552829 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:41.552847 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:41.552910 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:41.553165 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:41.553166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:41.553168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:41.553236 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.553239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.554262 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.554263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.555490 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.555492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.556404 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.556404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.556458 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.556461 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:04:41.556525 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:04:41.556559 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":1,"thread_start_us":38,"threads_started":1}
I20260620 23:04:41.557494 28696 tablet_service.cc:1467] Processing CreateTablet for tablet f9a054b279594f999dda4a8898411343 (DEFAULT_TABLE table=test-table [id=102dd84fe10541bab4197a28c3819033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:41.557633 28830 tablet_service.cc:1467] Processing CreateTablet for tablet f9a054b279594f999dda4a8898411343 (DEFAULT_TABLE table=test-table [id=102dd84fe10541bab4197a28c3819033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:41.557838 28696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a054b279594f999dda4a8898411343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:41.557935 28830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a054b279594f999dda4a8898411343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:41.557926 28562 tablet_service.cc:1467] Processing CreateTablet for tablet f9a054b279594f999dda4a8898411343 (DEFAULT_TABLE table=test-table [id=102dd84fe10541bab4197a28c3819033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:41.558213 28562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a054b279594f999dda4a8898411343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:41.560103 28910 tablet_bootstrap.cc:492] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: Bootstrap starting.
I20260620 23:04:41.560647 28910 tablet_bootstrap.cc:654] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:41.561069 28911 tablet_bootstrap.cc:492] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: Bootstrap starting.
I20260620 23:04:41.561159 28910 log.cc:824] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:41.561285 28912 tablet_bootstrap.cc:492] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba: Bootstrap starting.
I20260620 23:04:41.561715 28911 tablet_bootstrap.cc:654] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:41.561894 28912 tablet_bootstrap.cc:654] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:41.562209 28911 log.cc:824] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:41.562438 28912 log.cc:824] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:41.562909 28911 tablet_bootstrap.cc:492] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: No bootstrap required, opened a new log
I20260620 23:04:41.562989 28911 ts_tablet_manager.cc:1397] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:04:41.563198 28912 tablet_bootstrap.cc:492] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba: No bootstrap required, opened a new log
I20260620 23:04:41.563269 28912 ts_tablet_manager.cc:1397] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:04:41.563349 28910 tablet_bootstrap.cc:492] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: No bootstrap required, opened a new log
I20260620 23:04:41.563413 28910 ts_tablet_manager.cc:1397] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 23:04:41.564782 28912 raft_consensus.cc:348] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } }
I20260620 23:04:41.564939 28912 raft_consensus.cc:374] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:41.564986 28912 raft_consensus.cc:729] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9250e4aa9b94bff9e2f46cd397077ba, State: Initialized, Role: FOLLOWER
I20260620 23:04:41.564918 28911 raft_consensus.cc:348] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } }
I20260620 23:04:41.565080 28911 raft_consensus.cc:374] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:41.565120 28911 raft_consensus.cc:729] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b61e83593b94841a11bf413fbeceb3f, State: Initialized, Role: FOLLOWER
I20260620 23:04:41.565125 28912 consensus_queue.cc:260] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } }
I20260620 23:04:41.565258 28911 consensus_queue.cc:260] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } }
I20260620 23:04:41.565260 28910 raft_consensus.cc:348] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } }
I20260620 23:04:41.565402 28912 ts_tablet_manager.cc:1428] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:04:41.565434 28910 raft_consensus.cc:374] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:41.565481 28910 raft_consensus.cc:729] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf87b3d10e24554ae27f55b007c3570, State: Initialized, Role: FOLLOWER
I20260620 23:04:41.565517 28628 heartbeater.cc:503] Master 127.17.188.126:44551 was elected leader, sending a full tablet report...
I20260620 23:04:41.565543 28911 ts_tablet_manager.cc:1428] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:04:41.565620 28910 consensus_queue.cc:260] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } }
I20260620 23:04:41.565673 28896 heartbeater.cc:503] Master 127.17.188.126:44551 was elected leader, sending a full tablet report...
I20260620 23:04:41.565872 28910 ts_tablet_manager.cc:1428] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:04:41.565997 28762 heartbeater.cc:503] Master 127.17.188.126:44551 was elected leader, sending a full tablet report...
I20260620 23:04:41.622007 28918 raft_consensus.cc:482] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:04:41.622121 28918 raft_consensus.cc:504] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } }
I20260620 23:04:41.622447 28918 leader_election.cc:288] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b61e83593b94841a11bf413fbeceb3f (127.17.188.67:40501), f9250e4aa9b94bff9e2f46cd397077ba (127.17.188.65:42439)
I20260620 23:04:41.625164 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.622755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:54151 (local address 127.17.188.67:40501)
0620 23:04:41.622809 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.622812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.622819 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:04:41.622844 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.622845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.622853 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.622897 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:41.623093 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.623545 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.624433 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.624801 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.624903 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.624913 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.625058 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.625094 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 23:04:41.625172 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.622616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:40501 (local address 127.17.188.66:54151)
0620 23:04:41.622669 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:41.622681 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:41.622721 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:41.622951 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:41.622952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:41.622953 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:41.623021 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.623023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.623635 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.623636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.624341 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.624343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.624901 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.624902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.624985 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.624988 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:04:41.625053 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 23:04:41.625094 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 23:04:41.625598 28919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.622737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:42439 (local address 127.17.188.66:44303)
0620 23:04:41.622938 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:41.622948 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:41.623058 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:41.623232 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:41.623232 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:41.623234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:41.623310 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.623312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.623924 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.623925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.624656 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:41.624658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.625348 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.625349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:41.625415 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.625419 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:04:41.625507 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 23:04:41.625545 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":121,"threads_started":1}
I20260620 23:04:41.625571 28850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a054b279594f999dda4a8898411343" candidate_uuid: "bdf87b3d10e24554ae27f55b007c3570" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b61e83593b94841a11bf413fbeceb3f" is_pre_election: true
I20260620 23:04:41.625630 28909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.622833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:44303 (local address 127.17.188.65:42439)
0620 23:04:41.622895 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.622897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.623041 (+ 144us) server_negotiation.cc:407] Connection header received
0620 23:04:41.623136 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.623137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.623148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.623228 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:41.623393 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.623831 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.624763 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.625164 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.625288 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.625297 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.625523 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.625570 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:04:41.625738 28850 raft_consensus.cc:2454] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf87b3d10e24554ae27f55b007c3570 in term 0.
I20260620 23:04:41.625975 28649 leader_election.cc:302] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b61e83593b94841a11bf413fbeceb3f, bdf87b3d10e24554ae27f55b007c3570; no voters:
I20260620 23:04:41.626016 28582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a054b279594f999dda4a8898411343" candidate_uuid: "bdf87b3d10e24554ae27f55b007c3570" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" is_pre_election: true
I20260620 23:04:41.626119 28918 raft_consensus.cc:2781] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:04:41.626181 28918 raft_consensus.cc:482] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:04:41.626214 28918 raft_consensus.cc:3037] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:41.626199 28582 raft_consensus.cc:2454] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf87b3d10e24554ae27f55b007c3570 in term 0.
I20260620 23:04:41.626950 28918 raft_consensus.cc:504] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } }
I20260620 23:04:41.627104 28918 leader_election.cc:288] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [CANDIDATE]: Term 1 election: Requested vote from peers 1b61e83593b94841a11bf413fbeceb3f (127.17.188.67:40501), f9250e4aa9b94bff9e2f46cd397077ba (127.17.188.65:42439)
I20260620 23:04:41.627274 28850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a054b279594f999dda4a8898411343" candidate_uuid: "bdf87b3d10e24554ae27f55b007c3570" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b61e83593b94841a11bf413fbeceb3f"
I20260620 23:04:41.627323 28582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a054b279594f999dda4a8898411343" candidate_uuid: "bdf87b3d10e24554ae27f55b007c3570" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9250e4aa9b94bff9e2f46cd397077ba"
I20260620 23:04:41.627357 28850 raft_consensus.cc:3037] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:41.627424 28582 raft_consensus.cc:3037] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:41.628068 28850 raft_consensus.cc:2454] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf87b3d10e24554ae27f55b007c3570 in term 1.
I20260620 23:04:41.628185 28582 raft_consensus.cc:2454] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf87b3d10e24554ae27f55b007c3570 in term 1.
I20260620 23:04:41.628252 28649 leader_election.cc:302] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b61e83593b94841a11bf413fbeceb3f, bdf87b3d10e24554ae27f55b007c3570; no voters:
I20260620 23:04:41.628316 28918 raft_consensus.cc:2781] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:41.628453 28918 raft_consensus.cc:686] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 1 LEADER]: Becoming Leader. State: Replica: bdf87b3d10e24554ae27f55b007c3570, State: Running, Role: LEADER
I20260620 23:04:41.628551 28918 consensus_queue.cc:237] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } }
W20260620 23:04:41.628742 28763 tablet.cc:2369] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:41.629438 28435 catalog_manager.cc:5495] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf87b3d10e24554ae27f55b007c3570 (127.17.188.66). New cstate: current_term: 1 leader_uuid: "bdf87b3d10e24554ae27f55b007c3570" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdf87b3d10e24554ae27f55b007c3570" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 } health_report { overall_health: UNKNOWN } } }
I20260620 23:04:41.651458 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.649183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.17.188.66:42947)
0620 23:04:41.649232 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:41.649233 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:41.649244 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:04:41.649301 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:41.649301 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:41.649356 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:41.649442 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:41.649532 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.649963 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.650811 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:41.650989 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:41.651049 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:41.651173 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:41.651254 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:41.651305 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:41.651384 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 23:04:41.651397 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:04:41.653051 28582 raft_consensus.cc:1261] T f9a054b279594f999dda4a8898411343 P f9250e4aa9b94bff9e2f46cd397077ba [term 1 FOLLOWER]: Refusing update from remote peer bdf87b3d10e24554ae27f55b007c3570: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:41.653051 28850 raft_consensus.cc:1261] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [term 1 FOLLOWER]: Refusing update from remote peer bdf87b3d10e24554ae27f55b007c3570: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:41.653513 28920 consensus_queue.cc:1035] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b61e83593b94841a11bf413fbeceb3f" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:41.653663 28918 consensus_queue.cc:1035] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9250e4aa9b94bff9e2f46cd397077ba" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:41.655961 28923 mvcc.cc:205] Tried to move back new op lower bound from 7299058408048361472 to 7299058407951310848. Current Snapshot: MvccSnapshot[applied={T|T < 7299058408048361472}]
I20260620 23:04:41.656118 28924 mvcc.cc:205] Tried to move back new op lower bound from 7299058408048361472 to 7299058407951310848. Current Snapshot: MvccSnapshot[applied={T|T < 7299058408048361472}]
I20260620 23:04:41.656369 28925 mvcc.cc:205] Tried to move back new op lower bound from 7299058408048361472 to 7299058407951310848. Current Snapshot: MvccSnapshot[applied={T|T < 7299058408048361472}]
I20260620 23:04:41.658178 28436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:41.658288 28436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:41.659029 28436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c492278d9694f64a8e47fb92a7edd52: Sending DeleteTablet for 3 replicas of tablet f9a054b279594f999dda4a8898411343
I20260620 23:04:41.659363 28696 tablet_service.cc:1514] Processing DeleteTablet for tablet f9a054b279594f999dda4a8898411343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:41 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43288
I20260620 23:04:41.659484 28830 tablet_service.cc:1514] Processing DeleteTablet for tablet f9a054b279594f999dda4a8898411343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:41 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:47528
I20260620 23:04:41.659726 28934 tablet_replica.cc:330] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: stopping tablet replica
I20260620 23:04:41.659746 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 28499
I20260620 23:04:41.659807 28562 tablet_service.cc:1514] Processing DeleteTablet for tablet f9a054b279594f999dda4a8898411343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:41 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:33330
I20260620 23:04:41.660353 28933 tablet_replica.cc:330] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: stopping tablet replica
I20260620 23:04:41.660382 28934 raft_consensus.cc:2229] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:41.660463 28933 raft_consensus.cc:2229] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:04:41.660562 28933 raft_consensus.cc:2258] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:41.660640 28934 raft_consensus.cc:2258] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:41.660945 28933 ts_tablet_manager.cc:1905] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:41.661254 28934 ts_tablet_manager.cc:1905] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:41.662779 28933 ts_tablet_manager.cc:1918] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:41.662876 28933 log.cc:1196] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/f9a054b279594f999dda4a8898411343
I20260620 23:04:41.663168 28933 ts_tablet_manager.cc:1939] T f9a054b279594f999dda4a8898411343 P bdf87b3d10e24554ae27f55b007c3570: Deleting consensus metadata
I20260620 23:04:41.663378 28934 ts_tablet_manager.cc:1918] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:41.663575 28934 log.cc:1196] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/f9a054b279594f999dda4a8898411343
I20260620 23:04:41.663566 28422 catalog_manager.cc:4841] TS bdf87b3d10e24554ae27f55b007c3570 (127.17.188.66:42947): tablet f9a054b279594f999dda4a8898411343 (table test-table [id=102dd84fe10541bab4197a28c3819033]) successfully deleted
I20260620 23:04:41.663930 28934 ts_tablet_manager.cc:1939] T f9a054b279594f999dda4a8898411343 P 1b61e83593b94841a11bf413fbeceb3f: Deleting consensus metadata
I20260620 23:04:41.664357 28421 catalog_manager.cc:4841] TS 1b61e83593b94841a11bf413fbeceb3f (127.17.188.67:40501): tablet f9a054b279594f999dda4a8898411343 (table test-table [id=102dd84fe10541bab4197a28c3819033]) successfully deleted
W20260620 23:04:41.666671 28421 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:42439 (error 108)
I20260620 23:04:41.666903 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 28633
I20260620 23:04:41.667227 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:41.667109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:42439 (local address 127.0.0.1:33338)
0620 23:04:41.667152 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:41.667196 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:42439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 23:04:41.667330 28421 catalog_manager.cc:4584] TS f9250e4aa9b94bff9e2f46cd397077ba (127.17.188.65:42439): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f9a054b279594f999dda4a8898411343: Network error: Client connection negotiation failed: client connection to 127.17.188.65:42439: connect: Connection refused (error 111)
I20260620 23:04:41.672247 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 28767
I20260620 23:04:41.676571 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 28405
2026-06-20T23:04:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1910 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28938](info): set up 2 sockets
Jun 20 23:04:41 dist-test-slave-zpss krb5kdc[28938](info): commencing operation
krb5kdc: starting...
W20260620 23:04:42.705515 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:04:42 dist-test-slave-zpss krb5kdc[28938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.188.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.188.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/rangeradmin_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/rangeradmin_127.17.188.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.188.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.188.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/rangerlookup_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/rangerlookup_127.17.188.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.188.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.188.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/HTTP_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/HTTP_127.17.188.84@KRBTEST.COM.keytab.
I20260620 23:04:42.857508 18161 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRp9mku/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/postgres -l logfile start
2026-06-20 23:04:45.214 UTC [28968] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 23:04:45.214 UTC [28968] LOG: listening on IPv4 address "127.17.188.84", port 42617
2026-06-20 23:04:45.217 UTC [28968] LOG: listening on Unix socket "/tmp/.s.PGSQL.42617"
2026-06-20 23:04:45.238 UTC [28971] LOG: database system was shut down at 2026-06-20 23:04:43 UTC
2026-06-20 23:04:45.241 UTC [28968] LOG: database system is ready to accept connections
I20260620 23:04:46.189224 18161 mini_postgres.cc:96] Postgres bound to 42617
2026-06-20 23:04:46.196 UTC [28979] FATAL: database "slave" does not exist
127.17.188.84:42617 - accepting connections
I20260620 23:04:46.196635 18161 mini_ranger.cc:161] Starting Ranger...
I20260620 23:04:46.211014 18161 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 23:04:46.630730 18161 mini_ranger.cc:87] Created ranger Postgres database
I20260620 23:04:46.630797 18161 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-admin
2026-06-20 23:04:46,903 [I] DB FLAVOR :POSTGRES
2026-06-20 23:04:46,903 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:04:46,903 [I] Checking connection..
2026-06-20 23:04:46,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 23:04:47,134 [I] Checking connection passed.
2026-06-20 23:04:47,134 [I] --------- Verifying version history table ---------
2026-06-20 23:04:47,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:04:47,356 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:04:47,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:04:47,618 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:04:47,619 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 23:04:47,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 23:04:47.868 UTC [29043] WARNING: there is no transaction in progress
2026-06-20 23:04:47,874 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 23:04:47,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:04:48,105 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 23:04:48,105 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:04:48,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 23:04:48,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 23:04:48,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpss.c.gcp-upstream.internal','N') ;"
2026-06-20 23:04:48,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:04:49,054 [I] Table x_portal_user does not exist in database ranger
2026-06-20 23:04:49,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:04:49,300 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 23:04:49,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:04:49,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:04:49,770 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 23:04:49,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 23:04:50.658 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.672 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.687 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.701 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.716 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.731 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.787 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.793 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.807 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.816 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.826 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.839 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.849 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.854 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.860 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:50.866 UTC [29151] WARNING: there is no transaction in progress
2026-06-20 23:04:51,466 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 23:04:51,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 23:04:51,724 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 23:04:51,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:04:51,950 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 23:04:51,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:04:52,197 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 23:04:52,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:04:52,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:04:52,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpss.c.gcp-upstream.internal';"
2026-06-20 23:04:52,903 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 23:04:52,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 23:04:53,140 [I] DB_PATCHES have already been applied
I20260620 23:04:53.146013 18161 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRp9mku/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 23:04:53.146075 18161 mini_ranger.cc:193] Using host: 127.17.188.84
I20260620 23:04:53.148916 18161 mini_ranger.cc:227] Ranger admin URL: http://127.17.188.84:41935
Jun 20, 2026 11:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 11:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 11:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 11:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 11:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 23:04:54.230930 18161 mini_ranger.cc:160] Time spent starting Ranger: real 8.034s user 0.000s sys 0.009s
Jun 20, 2026 11:05:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 11:05:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 11:05:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 11:05:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 11:05:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 23:05:15.369325 18161 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.188.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.188.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/rangerkms_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/rangerkms_127.17.188.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.188.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.188.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.188.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/HTTP_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.188.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/HTTP_127.17.188.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 23:05:15 dist-test-slave-zpss krb5kdc[28938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996715, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 23:05:15.445230 18161 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 23:05:15.453794 18161 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 23:05:16.246272 18161 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 23:05:16.246361 18161 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-kms
I20260620 23:05:16.246379 18161 mini_ranger_kms.cc:227] Using postgres at 127.17.188.84:42617
2026-06-20 23:05:16,416 [I] DB FLAVOR :POSTGRES
2026-06-20 23:05:16,416 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:05:16,416 [I] Checking connection
2026-06-20 23:05:16,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 23:05:16,638 [I] connection success
2026-06-20 23:05:16,638 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 23:05:16,638 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 23:05:16,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 23:05:16,878 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 23:05:16,879 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:05:16,879 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 23:05:16,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:42617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 23:05:17,147 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 23:05:17.588212 18161 mini_ranger_kms.cc:310] Created kms service
I20260620 23:05:17.787334 18161 mini_ranger_kms.cc:326] Created kudu user
I20260620 23:05:17.828052 18161 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 23:05:17 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996715, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:05:18.194454 18161 mini_ranger_kms.cc:384] Added ranger policy
I20260620 23:05:18.194736 18161 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 23:05:18.194772 18161 mini_ranger_kms.cc:240] Using host: 127.17.188.84
I20260620 23:05:18.200238 18161 mini_ranger_kms.cc:276] Ranger KMS PID: 29331
I20260620 23:05:18.200356 18161 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.188.84:37585
Jun 20, 2026 11:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 11:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 11:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 11:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 20 23:05:20 dist-test-slave-zpss krb5kdc[28938](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996720, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.188.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:05:20.423174 18161 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.978s user 0.000s sys 0.154s
I20260620 23:05:20.423280 18161 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 23:05:20.423367 18161 mini_ranger_kms.cc:401] 127.17.188.84:37585/kms/v1/keys
Jun 20 23:05:20 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996720, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.188.84@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
Jun 20 23:05:20 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996720, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.188.84@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
Jun 20 23:05:23 dist-test-slave-zpss krb5kdc[28938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:05:23.078353 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:44827
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:44827
--ranger_config_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:37585/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:05:23.156013 29382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:05:23.156247 29382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:05:23.156284 29382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:05:23.157411 29382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:05:23.157486 29382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:05:23.157513 29382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:05:23.157544 29382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:05:23.157569 29382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:05:23.159068 29382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.188.84:37585/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:44827
--ranger_config_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:44827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:05:23.159366 29382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:05:23.159581 29382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:05:23.164589 29382 server_base.cc:1034] running on GCE node
W20260620 23:05:23.164698 29388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:05:23.164758 29390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:05:23.164716 29387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:05:23.165030 29382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:05:23.165307 29382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:05:23.166493 29382 hybrid_clock.cc:648] HybridClock initialized: now 1781996723166475 us; error 37 us; skew 500 ppm
Jun 20 23:05:23 dist-test-slave-zpss krb5kdc[28938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996723, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:05:23.171516 29382 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:05:23.171967 29382 webserver.cc:460] Webserver started at http://127.17.188.126:43169/ using document root <none> and password file <none>
I20260620 23:05:23.172168 29382 fs_manager.cc:359] Metadata directory not provided
I20260620 23:05:23.172219 29382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:05:23.172350 29382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:05:23 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996723, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:05:23.216014 29382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "6aefe3753c394e288d99f3fa81bd3e0c"
format_stamp: "Formatted at 2026-06-20 23:05:23 on dist-test-slave-zpss"
server_key: "843c751df61e473ee57df1ce84a3bbf4"
server_key_iv: "34aabda1d6e22f46a72f56f1d39dce25"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:23.216423 29382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "6aefe3753c394e288d99f3fa81bd3e0c"
format_stamp: "Formatted at 2026-06-20 23:05:23 on dist-test-slave-zpss"
server_key: "843c751df61e473ee57df1ce84a3bbf4"
server_key_iv: "34aabda1d6e22f46a72f56f1d39dce25"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:23.217877 29382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260620 23:05:23.228276 29397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:23.228477 29382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:05:23.228569 29382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "6aefe3753c394e288d99f3fa81bd3e0c"
format_stamp: "Formatted at 2026-06-20 23:05:23 on dist-test-slave-zpss"
server_key: "843c751df61e473ee57df1ce84a3bbf4"
server_key_iv: "34aabda1d6e22f46a72f56f1d39dce25"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:23.228681 29382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:05:23.239362 29382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:05:23.241204 29382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:05:23.241357 29382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:05:23.246091 29382 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:44827
I20260620 23:05:23.246120 29449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:44827 every 8 connection(s)
I20260620 23:05:23.246467 29382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:05:23.247105 29450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:05:23.249241 29450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c: Bootstrap starting.
I20260620 23:05:23.249816 29450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c: Neither blocks nor log segments found. Creating new log.
I20260620 23:05:23.250109 29450 log.cc:824] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:05:23.250808 29450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c: No bootstrap required, opened a new log
I20260620 23:05:23.253185 29450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefe3753c394e288d99f3fa81bd3e0c" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44827 } }
I20260620 23:05:23.253371 29450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:05:23.253386 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 29382
I20260620 23:05:23.253449 29450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aefe3753c394e288d99f3fa81bd3e0c, State: Initialized, Role: FOLLOWER
I20260620 23:05:23.253482 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:05:23.253680 18161 external_mini_cluster.cc:1428] Setting key ae165f37dc346d14cf57dbe4ae8991de
I20260620 23:05:23.253649 29450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefe3753c394e288d99f3fa81bd3e0c" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44827 } }
I20260620 23:05:23.253746 29450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:05:23.253800 29450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:05:23.253872 29450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:05:23.254577 29450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefe3753c394e288d99f3fa81bd3e0c" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44827 } }
I20260620 23:05:23.254695 29450 leader_election.cc:302] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6aefe3753c394e288d99f3fa81bd3e0c; no voters:
I20260620 23:05:23.254916 29450 leader_election.cc:288] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:05:23.254966 29455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:05:23.255179 29455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [term 1 LEADER]: Becoming Leader. State: Replica: 6aefe3753c394e288d99f3fa81bd3e0c, State: Running, Role: LEADER
I20260620 23:05:23.255239 29450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:05:23.256589 29455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefe3753c394e288d99f3fa81bd3e0c" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44827 } }
I20260620 23:05:23.257424 29456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6aefe3753c394e288d99f3fa81bd3e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefe3753c394e288d99f3fa81bd3e0c" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44827 } } }
I20260620 23:05:23.257508 29456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [sys.catalog]: This master's current role is: LEADER
I20260620 23:05:23.257684 29470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:05:23.258041 29450 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 23:05:23.258191 29470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:05:23.261847 29457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aefe3753c394e288d99f3fa81bd3e0c. Latest consensus state: current_term: 1 leader_uuid: "6aefe3753c394e288d99f3fa81bd3e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aefe3753c394e288d99f3fa81bd3e0c" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 44827 } } }
I20260620 23:05:23.262061 29457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c [sys.catalog]: This master's current role is: LEADER
Jun 20 23:05:23 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:05:23.264920 29470 catalog_manager.cc:1347] Generated new cluster ID: 0ddc1d7da0d144e9b12ab29e5b488a9f
I20260620 23:05:23.264982 29470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:05:23.268565 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:23.254114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42396 (local address 127.17.188.126:44827)
0620 23:05:23.254300 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:23.254309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:23.254340 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:05:23.254488 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:23.254495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:23.254549 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:23.254699 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:05:23.255639 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:23.257011 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:23.258340 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:23.261440 (+ 3100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:23.261637 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:23.264401 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:05:23.264406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:05:23.264407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:05:23.264418 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:05:23.266584 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:23.266985 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:23.266988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:23.266988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:23.267095 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:23.267296 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:23.267297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:23.267297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:23.267593 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:05:23.267753 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:23.268170 (+ 417us) server_negotiation.cc:299] Negotiation successful
0620 23:05:23.268294 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260620 23:05:23.277347 29470 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:05:23.278204 29470 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:05:23.287478 29470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c: Generated new TSK 0
I20260620 23:05:23.287706 29470 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:05:23.952646 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:37585/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44827
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:05:24.058511 29495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:05:24.058717 29495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:05:24.058740 29495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:05:24.060405 29495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:05:24.060463 29495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:05:24.060500 29495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:05:24.062515 29495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.188.84:37585/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:05:24.062741 29495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:05:24.062969 29495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:05:24.069067 29501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:05:24.069182 29503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:05:24.069049 29500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:05:24.069765 29495 server_base.cc:1034] running on GCE node
I20260620 23:05:24.069967 29495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:05:24.070225 29495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:05:24.071414 29495 hybrid_clock.cc:648] HybridClock initialized: now 1781996724071402 us; error 28 us; skew 500 ppm
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:05:24.073961 29495 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:05:24.074435 29495 webserver.cc:460] Webserver started at http://127.17.188.65:38077/ using document root <none> and password file <none>
I20260620 23:05:24.074673 29495 fs_manager.cc:359] Metadata directory not provided
I20260620 23:05:24.074716 29495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:05:24.074841 29495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:05:24.088477 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "99256177c13c473abcc494ea3639bd86"
format_stamp: "Formatted at 2026-06-20 23:05:24 on dist-test-slave-zpss"
server_key: "9a9ebe3f35c1ec471c832702c003c453"
server_key_iv: "6737c09e033334dc221e31c89e1f8448"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:24.088898 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "99256177c13c473abcc494ea3639bd86"
format_stamp: "Formatted at 2026-06-20 23:05:24 on dist-test-slave-zpss"
server_key: "9a9ebe3f35c1ec471c832702c003c453"
server_key_iv: "6737c09e033334dc221e31c89e1f8448"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:24.090680 29495 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 23:05:24.105593 29510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:24.105901 29495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 23:05:24.106045 29495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "99256177c13c473abcc494ea3639bd86"
format_stamp: "Formatted at 2026-06-20 23:05:24 on dist-test-slave-zpss"
server_key: "9a9ebe3f35c1ec471c832702c003c453"
server_key_iv: "6737c09e033334dc221e31c89e1f8448"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:24.106204 29495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:05:24.123934 29495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:05:24.126281 29495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:05:24.126428 29495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:05:24.126677 29495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:05:24.127017 29495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:05:24.127066 29495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:24.127110 29495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:05:24.127166 29495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:24.135166 29495 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:33147
I20260620 23:05:24.135612 29495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:05:24.136343 29625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:33147 every 8 connection(s)
I20260620 23:05:24.137534 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 29495
I20260620 23:05:24.137687 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:05:24.137909 18161 external_mini_cluster.cc:1428] Setting key b0b494151febc66d36a90d28ea29ee79
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
I20260620 23:05:24.147459 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.136338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:44827 (local address 127.17.188.65:53505)
0620 23:05:24.136558 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0620 23:05:24.136601 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 23:05:24.137050 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:05:24.137474 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:05:24.137478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:05:24.137748 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:05:24.138005 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.138030 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.138959 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.138960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.140292 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.140295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.140844 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.140845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.140989 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.141588 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:05:24.141598 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:05:24.143241 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:05:24.145572 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:05:24.145579 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:05:24.145581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:05:24.145921 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:05:24.146179 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:05:24.146180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:05:24.146180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:05:24.146343 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:05:24.146714 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:05:24.146722 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:05:24.146904 (+ 182us) client_negotiation.cc:769] Sending connection context
0620 23:05:24.147021 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 23:05:24.147182 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":82,"threads_started":1}
I20260620 23:05:24.147943 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.136325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:53505 (local address 127.17.188.126:44827)
0620 23:05:24.137226 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:24.137228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:24.137235 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:05:24.137258 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:24.137258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:24.137269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:24.137380 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:05:24.138139 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.138861 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.140409 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.140711 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.140779 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.143359 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:05:24.143361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:05:24.143362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:05:24.143369 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:05:24.145455 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:24.146020 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:24.146022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:24.146022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:24.146097 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:24.146435 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:24.146436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:24.146437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:24.146594 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:05:24.146670 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:24.147751 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0620 23:05:24.147857 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":102,"threads_started":1}
I20260620 23:05:24.148403 29626 heartbeater.cc:347] Connected to a master server at 127.17.188.126:44827
I20260620 23:05:24.148552 29626 heartbeater.cc:464] Registering TS with master...
I20260620 23:05:24.148784 29626 heartbeater.cc:511] Master 127.17.188.126:44827 requested a full tablet report, sending...
I20260620 23:05:24.149391 29414 ts_manager.cc:194] Registered new tserver with Master: 99256177c13c473abcc494ea3639bd86 (127.17.188.65:33147)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
I20260620 23:05:24.150079 29414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:53505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:05:24.165762 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:37585/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44827
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:05:24.275359 29632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:05:24.275612 29632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:05:24.275646 29632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:05:24.277381 29632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:05:24.277470 29632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:05:24.277523 29632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:05:24.279654 29632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.188.84:37585/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:05:24.279896 29632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:05:24.280228 29632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 23:05:24.286293 29642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:05:24.286873 29639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:05:24.286993 29640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:05:24.287978 29632 server_base.cc:1034] running on GCE node
I20260620 23:05:24.288193 29632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:05:24.288460 29632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:05:24.289628 29632 hybrid_clock.cc:648] HybridClock initialized: now 1781996724289611 us; error 29 us; skew 500 ppm
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:05:24.292800 29632 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:05:24.293283 29632 webserver.cc:460] Webserver started at http://127.17.188.66:40681/ using document root <none> and password file <none>
I20260620 23:05:24.293550 29632 fs_manager.cc:359] Metadata directory not provided
I20260620 23:05:24.293612 29632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:05:24.293730 29632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:05:24.316448 29632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "cb260eb513e248418fdd1844c8e64286"
format_stamp: "Formatted at 2026-06-20 23:05:24 on dist-test-slave-zpss"
server_key: "4345312388d5e45e2e3b41bd5fe27aee"
server_key_iv: "caa5ef1148009eaf2cff4244f06f78b7"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:24.317601 29632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "cb260eb513e248418fdd1844c8e64286"
format_stamp: "Formatted at 2026-06-20 23:05:24 on dist-test-slave-zpss"
server_key: "4345312388d5e45e2e3b41bd5fe27aee"
server_key_iv: "caa5ef1148009eaf2cff4244f06f78b7"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:24.325699 29632 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
I20260620 23:05:24.338151 29649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:24.338414 29632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:05:24.338500 29632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "cb260eb513e248418fdd1844c8e64286"
format_stamp: "Formatted at 2026-06-20 23:05:24 on dist-test-slave-zpss"
server_key: "4345312388d5e45e2e3b41bd5fe27aee"
server_key_iv: "caa5ef1148009eaf2cff4244f06f78b7"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:24.338583 29632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:05:24.350742 29632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:05:24.352833 29632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:05:24.352972 29632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:05:24.353201 29632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:05:24.353569 29632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:05:24.353619 29632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:24.353674 29632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:05:24.353700 29632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:24.360716 29632 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:39741
I20260620 23:05:24.360847 29763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:39741 every 8 connection(s)
I20260620 23:05:24.361131 29632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:05:24.363137 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 29632
I20260620 23:05:24.363234 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:05:24.363413 18161 external_mini_cluster.cc:1428] Setting key 696f1b09a2ffce7404116b9775c850c4
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:05:24.379416 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.361905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:45715 (local address 127.17.188.126:44827)
0620 23:05:24.361945 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:24.361948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:24.362173 (+ 225us) server_negotiation.cc:407] Connection header received
0620 23:05:24.362850 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:24.362852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:24.362869 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:24.362924 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:05:24.363688 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.364224 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.365792 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.366141 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.366186 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.375839 (+ 9653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:05:24.375841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:05:24.375841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:05:24.375850 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:05:24.377617 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:24.378162 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:24.378163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:24.378164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:24.378226 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:24.378561 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:24.378562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:24.378563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:24.378719 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:05:24.378785 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:24.379168 (+ 383us) server_negotiation.cc:299] Negotiation successful
0620 23:05:24.379267 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:05:24.380095 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.361903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:44827 (local address 127.17.188.66:45715)
0620 23:05:24.362056 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 23:05:24.362090 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:05:24.362725 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:05:24.362993 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:05:24.362997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:05:24.363254 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:05:24.363540 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.363547 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.364332 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.364333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.365686 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.365690 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.366274 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.366275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.366416 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.371202 (+ 4786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:05:24.371214 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:05:24.375701 (+ 4487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:05:24.377727 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:05:24.377736 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:05:24.377739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:05:24.378051 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:05:24.378299 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:05:24.378299 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:05:24.378300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:05:24.378453 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:05:24.378812 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:05:24.378819 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:05:24.379010 (+ 191us) client_negotiation.cc:769] Sending connection context
0620 23:05:24.379627 (+ 617us) client_negotiation.cc:240] Negotiation successful
0620 23:05:24.379802 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260620 23:05:24.380693 29764 heartbeater.cc:347] Connected to a master server at 127.17.188.126:44827
I20260620 23:05:24.380837 29764 heartbeater.cc:464] Registering TS with master...
I20260620 23:05:24.381090 29764 heartbeater.cc:511] Master 127.17.188.126:44827 requested a full tablet report, sending...
I20260620 23:05:24.381528 29414 ts_manager.cc:194] Registered new tserver with Master: cb260eb513e248418fdd1844c8e64286 (127.17.188.66:39741)
I20260620 23:05:24.382066 29414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:45715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:05:24.392004 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:37585/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44827
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:05:24.494940 29769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:05:24.495159 29769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:05:24.495199 29769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:05:24.496574 29769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:05:24.496635 29769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:05:24.496680 29769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:05:24.498333 29769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.188.84:37585/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:44827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:05:24.498585 29769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:05:24.498976 29769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:05:24.503999 29774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:05:24.504256 29777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:05:24.504065 29775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:05:24.504413 29769 server_base.cc:1034] running on GCE node
I20260620 23:05:24.504637 29769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:05:24.504923 29769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:05:24.506062 29769 hybrid_clock.cc:648] HybridClock initialized: now 1781996724506052 us; error 28 us; skew 500 ppm
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:05:24.509280 29769 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:05:24.509840 29769 webserver.cc:460] Webserver started at http://127.17.188.67:40431/ using document root <none> and password file <none>
I20260620 23:05:24.510169 29769 fs_manager.cc:359] Metadata directory not provided
I20260620 23:05:24.510283 29769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:05:24.510443 29769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:05:24.523861 29769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "35e40751b6de4f45a801aa16ede9da30"
format_stamp: "Formatted at 2026-06-20 23:05:24 on dist-test-slave-zpss"
server_key: "d674ccdaeb365a34764d95b19f2faff7"
server_key_iv: "fbc48d5435c17528b158943d0eb783fc"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:24.524322 29769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "35e40751b6de4f45a801aa16ede9da30"
format_stamp: "Formatted at 2026-06-20 23:05:24 on dist-test-slave-zpss"
server_key: "d674ccdaeb365a34764d95b19f2faff7"
server_key_iv: "fbc48d5435c17528b158943d0eb783fc"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:24.531047 29769 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260620 23:05:24.548992 29784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:24.549266 29769 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260620 23:05:24.549383 29769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "35e40751b6de4f45a801aa16ede9da30"
format_stamp: "Formatted at 2026-06-20 23:05:24 on dist-test-slave-zpss"
server_key: "d674ccdaeb365a34764d95b19f2faff7"
server_key_iv: "fbc48d5435c17528b158943d0eb783fc"
server_key_version: "kuduclusterkey@0"
I20260620 23:05:24.549471 29769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:05:24.568480 29769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:05:24.580471 29769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:05:24.580708 29769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:05:24.581012 29769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:05:24.581461 29769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:05:24.581555 29769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:24.581629 29769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:05:24.581686 29769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:05:24.590641 29769 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:46177
I20260620 23:05:24.591220 29769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:05:24.591933 29897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:46177 every 8 connection(s)
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:05:24.600215 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 29769
I20260620 23:05:24.600310 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:05:24.600536 18161 external_mini_cluster.cc:1428] Setting key fc5ee6f0c11c701e5c67bf9bb50585dd
I20260620 23:05:24.604072 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.592063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:37505 (local address 127.17.188.126:44827)
0620 23:05:24.592108 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:24.592110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:24.592363 (+ 253us) server_negotiation.cc:407] Connection header received
0620 23:05:24.593237 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:24.593238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:24.593253 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:24.593292 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:05:24.594293 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.595126 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.596785 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.597097 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.597146 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.600144 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:05:24.600145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:05:24.600146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:05:24.600154 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:05:24.602261 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:24.602830 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:24.602832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:24.602832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:24.602897 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:24.603210 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:24.603211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:24.603211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:24.603356 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:05:24.603408 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:24.603848 (+ 440us) server_negotiation.cc:299] Negotiation successful
0620 23:05:24.603943 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:05:24.605013 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.592059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:44827 (local address 127.17.188.67:37505)
0620 23:05:24.592225 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:05:24.592267 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:05:24.593087 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:05:24.593558 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:05:24.593570 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:05:24.593855 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:05:24.594163 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.594169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.595258 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.595259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.596655 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.596658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.597243 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.597244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.597411 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.598097 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:05:24.598116 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:05:24.600013 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:05:24.602386 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:05:24.602394 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:05:24.602396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:05:24.602716 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:05:24.602982 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:05:24.602983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:05:24.602983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:05:24.603120 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:05:24.603465 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:05:24.603470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:05:24.603650 (+ 180us) client_negotiation.cc:769] Sending connection context
0620 23:05:24.604422 (+ 772us) client_negotiation.cc:240] Negotiation successful
0620 23:05:24.604602 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260620 23:05:24.605883 29898 heartbeater.cc:347] Connected to a master server at 127.17.188.126:44827
I20260620 23:05:24.605991 29898 heartbeater.cc:464] Registering TS with master...
I20260620 23:05:24.606178 29898 heartbeater.cc:511] Master 127.17.188.126:44827 requested a full tablet report, sending...
I20260620 23:05:24.606595 29414 ts_manager.cc:194] Registered new tserver with Master: 35e40751b6de4f45a801aa16ede9da30 (127.17.188.67:46177)
I20260620 23:05:24.607112 29414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:37505
I20260620 23:05:24.611994 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:05:24.629197 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.618232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42408 (local address 127.17.188.126:44827)
0620 23:05:24.618294 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:24.618296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:24.618512 (+ 216us) server_negotiation.cc:407] Connection header received
0620 23:05:24.618623 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:24.618624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:24.618637 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:24.618670 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:05:24.619502 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.620169 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.621374 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.621631 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.621674 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.625964 (+ 4290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:05:24.625966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:05:24.625966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:05:24.625973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:05:24.627843 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:24.628252 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:24.628253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:24.628254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:24.628327 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:05:24.628569 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:05:24.628570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:05:24.628570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:05:24.628715 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:05:24.628792 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:24.629029 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 23:05:24.629104 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 23:05:24.632675 29414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:05:24.635145 29414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:05:24.676905 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.663876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:33147 (local address 127.0.0.1:46296)
0620 23:05:24.669664 (+ 5788us) negotiation.cc:107] Waiting for socket to connect
0620 23:05:24.669674 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:05:24.669711 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:05:24.670105 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:05:24.670108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:05:24.670111 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:05:24.670191 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.670193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.671778 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.671779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.675248 (+ 3469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.675251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.676601 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.676602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.676690 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.676706 (+ 16us) client_negotiation.cc:769] Sending connection context
0620 23:05:24.676774 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:05:24.676825 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5773,"thread_start_us":63,"threads_started":1}
I20260620 23:05:24.677498 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.664008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.17.188.65:33147)
0620 23:05:24.669854 (+ 5846us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:24.669860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:24.669871 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:05:24.669909 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:24.669912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:24.669928 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:24.670009 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:05:24.670298 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.671427 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.675347 (+ 3920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.676437 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.676509 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.676567 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:24.677379 (+ 812us) server_negotiation.cc:299] Negotiation successful
0620 23:05:24.677421 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5806,"thread_start_us":59,"threads_started":1}
I20260620 23:05:24.678333 29560 tablet_service.cc:1467] Processing CreateTablet for tablet 0cdaa94fdf5e494da27664ad9e8f8d85 (DEFAULT_TABLE table=test-table [id=08861f9651934669aed9a1864af44fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:05:24.678656 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cdaa94fdf5e494da27664ad9e8f8d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:05:24.681115 29917 tablet_bootstrap.cc:492] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86: Bootstrap starting.
I20260620 23:05:24.682593 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.663649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:39741 (local address 127.0.0.1:53266)
0620 23:05:24.669417 (+ 5768us) negotiation.cc:107] Waiting for socket to connect
0620 23:05:24.669465 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 23:05:24.669543 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:05:24.671839 (+ 2296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:05:24.671840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:05:24.671842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:05:24.671891 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.671893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.673109 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.673110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.674365 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.674368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.682357 (+ 7989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.682358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.682422 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.682426 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:05:24.682480 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 23:05:24.682518 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5726,"thread_start_us":82,"threads_started":1}
I20260620 23:05:24.684602 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.664358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.17.188.66:39741)
0620 23:05:24.671603 (+ 7245us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:24.671608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:24.671618 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:05:24.671656 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:24.671658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:24.671674 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:24.671748 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:05:24.671975 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.673005 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.681411 (+ 8406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.682170 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.682250 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.682318 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:24.684470 (+ 2152us) server_negotiation.cc:299] Negotiation successful
0620 23:05:24.684514 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7207,"thread_start_us":80,"threads_started":1}
I20260620 23:05:24.685127 29917 tablet_bootstrap.cc:654] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86: Neither blocks nor log segments found. Creating new log.
I20260620 23:05:24.685562 29917 log.cc:824] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86: Log is configured to *not* fsync() on all Append() calls
I20260620 23:05:24.685463 29698 tablet_service.cc:1467] Processing CreateTablet for tablet 0cdaa94fdf5e494da27664ad9e8f8d85 (DEFAULT_TABLE table=test-table [id=08861f9651934669aed9a1864af44fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:05:24.685820 29698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cdaa94fdf5e494da27664ad9e8f8d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:05:24.691679 29919 tablet_bootstrap.cc:492] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: Bootstrap starting.
I20260620 23:05:24.691723 29911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.664019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:46177 (local address 127.0.0.1:51354)
0620 23:05:24.673381 (+ 9362us) negotiation.cc:107] Waiting for socket to connect
0620 23:05:24.673392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:05:24.673443 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:05:24.683052 (+ 9609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:05:24.683053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:05:24.683055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:05:24.683092 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.683094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.689257 (+ 6163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.689258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.690419 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:24.690422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.691480 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.691481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:24.691546 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.691552 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:05:24.691612 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 23:05:24.691652 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9345,"thread_start_us":62,"threads_started":1}
I20260620 23:05:24.692013 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:24.664517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.17.188.67:46177)
0620 23:05:24.682864 (+ 18347us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:24.682868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:24.682879 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:05:24.682919 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:24.682921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:24.682938 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:24.683172 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:05:24.683257 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.684357 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.690578 (+ 6221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:24.691384 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:24.691471 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:24.691538 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:24.691887 (+ 349us) server_negotiation.cc:299] Negotiation successful
0620 23:05:24.691932 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18304,"thread_start_us":83,"threads_started":1}
I20260620 23:05:24.692724 29832 tablet_service.cc:1467] Processing CreateTablet for tablet 0cdaa94fdf5e494da27664ad9e8f8d85 (DEFAULT_TABLE table=test-table [id=08861f9651934669aed9a1864af44fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:05:24.692976 29919 tablet_bootstrap.cc:654] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: Neither blocks nor log segments found. Creating new log.
I20260620 23:05:24.693188 29832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cdaa94fdf5e494da27664ad9e8f8d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:05:24.693256 29919 log.cc:824] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: Log is configured to *not* fsync() on all Append() calls
I20260620 23:05:24.699992 29919 tablet_bootstrap.cc:492] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: No bootstrap required, opened a new log
I20260620 23:05:24.700075 29919 ts_tablet_manager.cc:1397] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260620 23:05:24.702421 29919 raft_consensus.cc:348] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } }
I20260620 23:05:24.702596 29919 raft_consensus.cc:374] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:05:24.702630 29919 raft_consensus.cc:729] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb260eb513e248418fdd1844c8e64286, State: Initialized, Role: FOLLOWER
I20260620 23:05:24.702775 29919 consensus_queue.cc:260] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } }
I20260620 23:05:24.703039 29919 ts_tablet_manager.cc:1428] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:05:24.703080 29764 heartbeater.cc:503] Master 127.17.188.126:44827 was elected leader, sending a full tablet report...
I20260620 23:05:24.703496 29917 tablet_bootstrap.cc:492] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86: No bootstrap required, opened a new log
I20260620 23:05:24.703696 29917 ts_tablet_manager.cc:1397] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.001s
I20260620 23:05:24.706435 29917 raft_consensus.cc:348] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } }
I20260620 23:05:24.706616 29917 raft_consensus.cc:374] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:05:24.706652 29917 raft_consensus.cc:729] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99256177c13c473abcc494ea3639bd86, State: Initialized, Role: FOLLOWER
I20260620 23:05:24.706787 29917 consensus_queue.cc:260] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } }
I20260620 23:05:24.707056 29917 ts_tablet_manager.cc:1428] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 23:05:24.709825 29626 heartbeater.cc:503] Master 127.17.188.126:44827 was elected leader, sending a full tablet report...
I20260620 23:05:24.710613 29921 tablet_bootstrap.cc:492] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: Bootstrap starting.
I20260620 23:05:24.711762 29921 tablet_bootstrap.cc:654] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: Neither blocks nor log segments found. Creating new log.
I20260620 23:05:24.712107 29921 log.cc:824] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: Log is configured to *not* fsync() on all Append() calls
I20260620 23:05:24.717770 29921 tablet_bootstrap.cc:492] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: No bootstrap required, opened a new log
I20260620 23:05:24.717900 29921 ts_tablet_manager.cc:1397] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260620 23:05:24.725049 29921 raft_consensus.cc:348] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } }
I20260620 23:05:24.726002 29921 raft_consensus.cc:374] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:05:24.726083 29921 raft_consensus.cc:729] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35e40751b6de4f45a801aa16ede9da30, State: Initialized, Role: FOLLOWER
I20260620 23:05:24.726296 29921 consensus_queue.cc:260] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } }
I20260620 23:05:24.726682 29921 ts_tablet_manager.cc:1428] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260620 23:05:24.726840 29898 heartbeater.cc:503] Master 127.17.188.126:44827 was elected leader, sending a full tablet report...
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
W20260620 23:05:24.842109 29899 tablet.cc:2369] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:05:24.861922 29765 tablet.cc:2369] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:05:24.886875 29627 tablet.cc:2369] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 23:05:24 dist-test-slave-zpss krb5kdc[28938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:05:25.068512 29923 raft_consensus.cc:482] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:05:25.068668 29923 raft_consensus.cc:504] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } }
I20260620 23:05:25.069051 29923 leader_election.cc:288] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb260eb513e248418fdd1844c8e64286 (127.17.188.66:39741), 35e40751b6de4f45a801aa16ede9da30 (127.17.188.67:46177)
I20260620 23:05:25.080790 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:25.069788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:38425 (local address 127.17.188.67:46177)
0620 23:05:25.069828 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:25.069831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:25.077374 (+ 7543us) server_negotiation.cc:407] Connection header received
0620 23:05:25.077483 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:25.077483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:25.077493 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:25.077533 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:05:25.077749 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.078484 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.079875 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.080399 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.080455 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:25.080463 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:25.080682 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 23:05:25.080718 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:05:25.081043 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:25.069539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:46177 (local address 127.17.188.65:38425)
0620 23:05:25.077288 (+ 7749us) negotiation.cc:107] Waiting for socket to connect
0620 23:05:25.077299 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:05:25.077401 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:05:25.077612 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:05:25.077613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:05:25.077614 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:05:25.077684 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:25.077685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.078595 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.078596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:25.079775 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:25.079778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.080519 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.080520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:25.080607 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:25.080610 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:05:25.080924 (+ 314us) client_negotiation.cc:240] Negotiation successful
0620 23:05:25.080966 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7731,"thread_start_us":53,"threads_started":1}
I20260620 23:05:25.081382 29852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cdaa94fdf5e494da27664ad9e8f8d85" candidate_uuid: "99256177c13c473abcc494ea3639bd86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35e40751b6de4f45a801aa16ede9da30" is_pre_election: true
I20260620 23:05:25.081557 29852 raft_consensus.cc:2454] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99256177c13c473abcc494ea3639bd86 in term 0.
I20260620 23:05:25.081589 29928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:25.069282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:39741 (local address 127.17.188.65:37711)
0620 23:05:25.075743 (+ 6461us) negotiation.cc:107] Waiting for socket to connect
0620 23:05:25.075754 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:05:25.075852 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:05:25.076048 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:05:25.076049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:05:25.076064 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:05:25.076145 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:25.076147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.077077 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.077077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:25.078392 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:05:25.078395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.081283 (+ 2888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.081284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:05:25.081370 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:25.081373 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:05:25.081453 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 23:05:25.081490 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6440,"thread_start_us":141,"threads_started":1}
I20260620 23:05:25.081583 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:25.069733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:37711 (local address 127.17.188.66:39741)
0620 23:05:25.069795 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:25.069798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:25.075844 (+ 6046us) server_negotiation.cc:407] Connection header received
0620 23:05:25.075911 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:25.075913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:25.075926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:25.075993 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:05:25.076257 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.076976 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.080610 (+ 3634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.081146 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.081199 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:25.081207 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:25.081464 (+ 257us) server_negotiation.cc:299] Negotiation successful
0620 23:05:25.081503 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:05:25.081799 29513 leader_election.cc:302] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35e40751b6de4f45a801aa16ede9da30, 99256177c13c473abcc494ea3639bd86; no voters:
I20260620 23:05:25.081868 29718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cdaa94fdf5e494da27664ad9e8f8d85" candidate_uuid: "99256177c13c473abcc494ea3639bd86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb260eb513e248418fdd1844c8e64286" is_pre_election: true
I20260620 23:05:25.081938 29923 raft_consensus.cc:2781] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:05:25.081992 29923 raft_consensus.cc:482] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:05:25.082015 29923 raft_consensus.cc:3037] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:05:25.082019 29718 raft_consensus.cc:2454] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99256177c13c473abcc494ea3639bd86 in term 0.
I20260620 23:05:25.082939 29923 raft_consensus.cc:504] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } }
I20260620 23:05:25.083072 29923 leader_election.cc:288] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [CANDIDATE]: Term 1 election: Requested vote from peers cb260eb513e248418fdd1844c8e64286 (127.17.188.66:39741), 35e40751b6de4f45a801aa16ede9da30 (127.17.188.67:46177)
I20260620 23:05:25.083276 29718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cdaa94fdf5e494da27664ad9e8f8d85" candidate_uuid: "99256177c13c473abcc494ea3639bd86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb260eb513e248418fdd1844c8e64286"
I20260620 23:05:25.083348 29718 raft_consensus.cc:3037] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:05:25.084180 29718 raft_consensus.cc:2454] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99256177c13c473abcc494ea3639bd86 in term 1.
I20260620 23:05:25.084293 29852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cdaa94fdf5e494da27664ad9e8f8d85" candidate_uuid: "99256177c13c473abcc494ea3639bd86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35e40751b6de4f45a801aa16ede9da30"
I20260620 23:05:25.084362 29852 raft_consensus.cc:3037] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:05:25.085215 29852 raft_consensus.cc:2454] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99256177c13c473abcc494ea3639bd86 in term 1.
I20260620 23:05:25.085307 29513 leader_election.cc:302] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99256177c13c473abcc494ea3639bd86, cb260eb513e248418fdd1844c8e64286; no voters:
I20260620 23:05:25.085484 29923 raft_consensus.cc:2781] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:05:25.085613 29923 raft_consensus.cc:686] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [term 1 LEADER]: Becoming Leader. State: Replica: 99256177c13c473abcc494ea3639bd86, State: Running, Role: LEADER
I20260620 23:05:25.085718 29923 consensus_queue.cc:237] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } }
I20260620 23:05:25.086563 29414 catalog_manager.cc:5495] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99256177c13c473abcc494ea3639bd86 (127.17.188.65). New cstate: current_term: 1 leader_uuid: "99256177c13c473abcc494ea3639bd86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99256177c13c473abcc494ea3639bd86" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260620 23:05:25.107954 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:25.104713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46310 (local address 127.17.188.65:33147)
0620 23:05:25.104754 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:05:25.104756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:05:25.104795 (+ 39us) server_negotiation.cc:407] Connection header received
0620 23:05:25.104829 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:05:25.104830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:05:25.104851 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:05:25.104915 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:05:25.105087 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.105787 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.106794 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:05:25.107087 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:05:25.107146 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:05:25.107410 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:05:25.107539 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:05:25.107619 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:05:25.107841 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 23:05:25.107860 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:05:25.112890 29852 raft_consensus.cc:1261] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [term 1 FOLLOWER]: Refusing update from remote peer 99256177c13c473abcc494ea3639bd86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:05:25.113212 29718 raft_consensus.cc:1261] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [term 1 FOLLOWER]: Refusing update from remote peer 99256177c13c473abcc494ea3639bd86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:05:25.113684 29923 consensus_queue.cc:1035] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb260eb513e248418fdd1844c8e64286" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 39741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:05:25.114099 29923 consensus_queue.cc:1035] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 99256177c13c473abcc494ea3639bd86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35e40751b6de4f45a801aa16ede9da30" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:05:25.124809 29933 mvcc.cc:205] Tried to move back new op lower bound from 7299058586054270976 to 7299058585951813632. Current Snapshot: MvccSnapshot[applied={T|T < 7299058586054270976}]
I20260620 23:05:25.125010 29934 mvcc.cc:205] Tried to move back new op lower bound from 7299058586054270976 to 7299058585951813632. Current Snapshot: MvccSnapshot[applied={T|T < 7299058586054270976}]
I20260620 23:05:25.125144 29936 mvcc.cc:205] Tried to move back new op lower bound from 7299058586054270976 to 7299058585951813632. Current Snapshot: MvccSnapshot[applied={T|T < 7299058586054270976}]
I20260620 23:05:25.128166 29414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:05:25.128257 29414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:05:25.129110 29414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6aefe3753c394e288d99f3fa81bd3e0c: Sending DeleteTablet for 3 replicas of tablet 0cdaa94fdf5e494da27664ad9e8f8d85
I20260620 23:05:25.129451 29698 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cdaa94fdf5e494da27664ad9e8f8d85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:05:25 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:53266
I20260620 23:05:25.130080 29560 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cdaa94fdf5e494da27664ad9e8f8d85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:05:25 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:46296
I20260620 23:05:25.130080 29832 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cdaa94fdf5e494da27664ad9e8f8d85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:05:25 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:51354
I20260620 23:05:25.130630 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 29495
I20260620 23:05:25.134627 29942 tablet_replica.cc:330] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: stopping tablet replica
I20260620 23:05:25.134794 29942 raft_consensus.cc:2229] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:05:25.134984 29942 raft_consensus.cc:2258] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:05:25.135504 29942 ts_tablet_manager.cc:1905] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:05:25.139367 29943 tablet_replica.cc:330] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: stopping tablet replica
I20260620 23:05:25.139551 29943 raft_consensus.cc:2229] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:05:25.139722 29943 raft_consensus.cc:2258] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:05:25.140017 29942 ts_tablet_manager.cc:1918] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:05:25.140098 29942 log.cc:1196] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/0cdaa94fdf5e494da27664ad9e8f8d85
I20260620 23:05:25.140395 29942 ts_tablet_manager.cc:1939] T 0cdaa94fdf5e494da27664ad9e8f8d85 P cb260eb513e248418fdd1844c8e64286: Deleting consensus metadata
I20260620 23:05:25.140620 29943 ts_tablet_manager.cc:1905] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:05:25.140863 29399 catalog_manager.cc:4841] TS cb260eb513e248418fdd1844c8e64286 (127.17.188.66:39741): tablet 0cdaa94fdf5e494da27664ad9e8f8d85 (table test-table [id=08861f9651934669aed9a1864af44fbc]) successfully deleted
I20260620 23:05:25.143733 29943 ts_tablet_manager.cc:1918] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:05:25.144188 29943 log.cc:1196] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/0cdaa94fdf5e494da27664ad9e8f8d85
I20260620 23:05:25.144531 29943 ts_tablet_manager.cc:1939] T 0cdaa94fdf5e494da27664ad9e8f8d85 P 35e40751b6de4f45a801aa16ede9da30: Deleting consensus metadata
W20260620 23:05:25.144894 29399 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:33147 (error 108)
I20260620 23:05:25.145895 29399 catalog_manager.cc:4841] TS 35e40751b6de4f45a801aa16ede9da30 (127.17.188.67:46177): tablet 0cdaa94fdf5e494da27664ad9e8f8d85 (table test-table [id=08861f9651934669aed9a1864af44fbc]) successfully deleted
I20260620 23:05:25.146301 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 29632
I20260620 23:05:25.151448 29911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:05:25.151290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:33147 (local address 127.0.0.1:46314)
0620 23:05:25.151341 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 23:05:25.151392 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260620 23:05:25.151588 29399 catalog_manager.cc:4584] TS 99256177c13c473abcc494ea3639bd86 (127.17.188.65:33147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0cdaa94fdf5e494da27664ad9e8f8d85: Network error: Client connection negotiation failed: client connection to 127.17.188.65:33147: connect: Connection refused (error 111)
I20260620 23:05:25.152670 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 29769
I20260620 23:05:25.158296 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 29382
I20260620 23:05:25.163872 18161 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 23:05:25.508006 18161 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 23:05:25.508101 18161 mini_ranger.cc:66] Stopping Ranger...
I20260620 23:05:25.852845 18161 mini_ranger.cc:68] Stopped Ranger
2026-06-20 23:05:25.853 UTC [28968] LOG: received smart shutdown request
2026-06-20 23:05:25.856 UTC [28968] LOG: background worker "logical replication launcher" (PID 28977) exited with exit code 1
2026-06-20 23:05:25.856 UTC [28972] LOG: shutting down
2026-06-20 23:05:26.005 UTC [28968] LOG: database system is shut down
2026-06-20T23:05:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44437 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:05:26 dist-test-slave-zpss krb5kdc[29954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:05:26 dist-test-slave-zpss krb5kdc[29954](info): set up 2 sockets
Jun 20 23:05:26 dist-test-slave-zpss krb5kdc[29954](info): commencing operation
krb5kdc: starting...
W20260620 23:05:27.149279 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:05:27 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:05:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:05:27Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.188.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.188.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/rangeradmin_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/rangeradmin_127.17.188.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.188.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.188.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/rangerlookup_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/rangerlookup_127.17.188.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.188.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.188.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/HTTP_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/HTTP_127.17.188.84@KRBTEST.COM.keytab.
I20260620 23:05:27.303951 18161 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRp9mku/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/postgres -l logfile start
2026-06-20 23:05:29.622 UTC [29984] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 23:05:29.622 UTC [29984] LOG: listening on IPv4 address "127.17.188.84", port 52471
2026-06-20 23:05:29.625 UTC [29984] LOG: listening on Unix socket "/tmp/.s.PGSQL.52471"
2026-06-20 23:05:29.646 UTC [29987] LOG: database system was shut down at 2026-06-20 23:05:28 UTC
2026-06-20 23:05:29.649 UTC [29984] LOG: database system is ready to accept connections
I20260620 23:05:30.597131 18161 mini_postgres.cc:96] Postgres bound to 52471
2026-06-20 23:05:30.603 UTC [29995] FATAL: database "slave" does not exist
127.17.188.84:52471 - accepting connections
I20260620 23:05:30.603649 18161 mini_ranger.cc:161] Starting Ranger...
I20260620 23:05:30.617746 18161 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 23:05:31.075300 18161 mini_ranger.cc:87] Created ranger Postgres database
I20260620 23:05:31.075373 18161 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-admin
2026-06-20 23:05:31,339 [I] DB FLAVOR :POSTGRES
2026-06-20 23:05:31,340 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:05:31,340 [I] Checking connection..
2026-06-20 23:05:31,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 23:05:31,593 [I] Checking connection passed.
2026-06-20 23:05:31,593 [I] --------- Verifying version history table ---------
2026-06-20 23:05:31,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:05:31,826 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:05:31,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:05:32,054 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:05:32,055 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 23:05:32,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 23:05:32.285 UTC [30059] WARNING: there is no transaction in progress
2026-06-20 23:05:32,291 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 23:05:32,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:05:32,529 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 23:05:32,530 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:05:32,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 23:05:32,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 23:05:33,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpss.c.gcp-upstream.internal','N') ;"
2026-06-20 23:05:33,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:05:33,517 [I] Table x_portal_user does not exist in database ranger
2026-06-20 23:05:33,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:05:33,756 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 23:05:33,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:05:33,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:05:34,247 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 23:05:34,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 23:05:35.115 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.129 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.145 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.159 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.174 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.190 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.251 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.257 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.270 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.280 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.290 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.303 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.312 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.318 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.324 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35.330 UTC [30167] WARNING: there is no transaction in progress
2026-06-20 23:05:35,901 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 23:05:35,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 23:05:36,130 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 23:05:36,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:05:36,352 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 23:05:36,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:05:36,631 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 23:05:36,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:05:36,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:05:37,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpss.c.gcp-upstream.internal';"
2026-06-20 23:05:37,301 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 23:05:37,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 23:05:37,554 [I] DB_PATCHES have already been applied
I20260620 23:05:37.559595 18161 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRp9mku/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 23:05:37.559651 18161 mini_ranger.cc:193] Using host: 127.17.188.84
I20260620 23:05:37.562646 18161 mini_ranger.cc:227] Ranger admin URL: http://127.17.188.84:54611
Jun 20, 2026 11:05:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:05:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:05:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 11:05:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 11:05:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 11:05:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 11:05:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 23:05:38.671736 18161 mini_ranger.cc:160] Time spent starting Ranger: real 8.068s user 0.001s sys 0.006s
Jun 20, 2026 11:05:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 11:05:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 20, 2026 11:05:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 11:05:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 11:05:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 23:05:59.834758 18161 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.188.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.188.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/rangerkms_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.188.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/rangerkms_127.17.188.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.188.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.188.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.188.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/HTTP_127.17.188.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.188.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/HTTP_127.17.188.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 23:05:59 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996759, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 23:05:59.923408 18161 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 23:05:59.932309 18161 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 23:06:00.668241 18161 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 23:06:00.668330 18161 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-kms
I20260620 23:06:00.668349 18161 mini_ranger_kms.cc:227] Using postgres at 127.17.188.84:52471
2026-06-20 23:06:00,827 [I] DB FLAVOR :POSTGRES
2026-06-20 23:06:00,828 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:06:00,828 [I] Checking connection
2026-06-20 23:06:00,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 23:06:01,068 [I] connection success
2026-06-20 23:06:01,069 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 23:06:01,069 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 23:06:01,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 23:06:01,303 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 23:06:01,303 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:06:01,304 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 23:06:01,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.188.84:52471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 23:06:01,551 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 23:06:02.033747 18161 mini_ranger_kms.cc:310] Created kms service
I20260620 23:06:02.222704 18161 mini_ranger_kms.cc:326] Created kudu user
I20260620 23:06:02.265666 18161 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 23:06:02 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996759, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:02.650311 18161 mini_ranger_kms.cc:384] Added ranger policy
I20260620 23:06:02.650430 18161 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRp9mku/build/release/bin/postgresql.jar:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRp9mku/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 23:06:02.650450 18161 mini_ranger_kms.cc:240] Using host: 127.17.188.84
I20260620 23:06:02.653650 18161 mini_ranger_kms.cc:276] Ranger KMS PID: 30347
I20260620 23:06:02.653736 18161 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.188.84:35453
Jun 20, 2026 11:06:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:06:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:06:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 11:06:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 11:06:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 11:06:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 20 23:06:04 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996764, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.188.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:04.877175 18161 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.954s user 0.006s sys 0.137s
I20260620 23:06:04.877286 18161 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 23:06:04.877362 18161 mini_ranger_kms.cc:401] 127.17.188.84:35453/kms/v1/keys
Jun 20 23:06:05 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996764, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.188.84@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
Jun 20 23:06:05 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996764, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.188.84@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
Jun 20 23:06:07 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:06:07.776439 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34301
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:34301
--ranger_config_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:35453/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:07.858606 30398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:07.858794 30398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:07.858814 30398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:07.859906 30398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:07.859944 30398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:07.859956 30398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:07.859977 30398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:07.859987 30398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:07.861510 30398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.188.84:35453/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:34301
--ranger_config_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:07.861732 30398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:07.861964 30398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:07.867417 30404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:07.867439 30406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:07.867576 30398 server_base.cc:1034] running on GCE node
W20260620 23:06:07.867417 30403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:07.867929 30398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:07.868211 30398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:07.869374 30398 hybrid_clock.cc:648] HybridClock initialized: now 1781996767869360 us; error 31 us; skew 500 ppm
Jun 20 23:06:07 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:07.871835 30398 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:06:07.872207 30398 webserver.cc:460] Webserver started at http://127.17.188.126:42065/ using document root <none> and password file <none>
I20260620 23:06:07.872406 30398 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:07.872442 30398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:07.872567 30398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:06:07 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:07.924033 30398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "2a34390249924f8596c5f181e7a2ef2e"
format_stamp: "Formatted at 2026-06-20 23:06:07 on dist-test-slave-zpss"
server_key: "88900b267cf869e3d9932784ca93118c"
server_key_iv: "54fac0d2a26db84f80fa100b64c2cfc9"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:07.924513 30398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "2a34390249924f8596c5f181e7a2ef2e"
format_stamp: "Formatted at 2026-06-20 23:06:07 on dist-test-slave-zpss"
server_key: "88900b267cf869e3d9932784ca93118c"
server_key_iv: "54fac0d2a26db84f80fa100b64c2cfc9"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:07.926472 30398 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 23:06:07.937281 30413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:07.938977 30398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 23:06:07.939086 30398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "2a34390249924f8596c5f181e7a2ef2e"
format_stamp: "Formatted at 2026-06-20 23:06:07 on dist-test-slave-zpss"
server_key: "88900b267cf869e3d9932784ca93118c"
server_key_iv: "54fac0d2a26db84f80fa100b64c2cfc9"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:07.939198 30398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:07.950295 30398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:07.952317 30398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:07.952467 30398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:07.957300 30398 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:34301
I20260620 23:06:07.957397 30465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:34301 every 8 connection(s)
I20260620 23:06:07.957715 30398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:06:07.958405 30466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:07.960539 30466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Bootstrap starting.
I20260620 23:06:07.961135 30466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:07.961503 30466 log.cc:824] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:07.961724 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 30398
I20260620 23:06:07.961823 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:06:07.961998 18161 external_mini_cluster.cc:1428] Setting key a2ba210c56d243c9f3b90daee0b93ba6
I20260620 23:06:07.962903 30466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: No bootstrap required, opened a new log
I20260620 23:06:07.965409 30466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } }
I20260620 23:06:07.965567 30466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:07.965596 30466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a34390249924f8596c5f181e7a2ef2e, State: Initialized, Role: FOLLOWER
I20260620 23:06:07.965699 30466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } }
I20260620 23:06:07.965758 30466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:06:07.965785 30466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:06:07.965829 30466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:07.966336 30466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } }
I20260620 23:06:07.966415 30466 leader_election.cc:302] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a34390249924f8596c5f181e7a2ef2e; no voters:
I20260620 23:06:07.966569 30466 leader_election.cc:288] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:06:07.966598 30471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:07.966701 30471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 1 LEADER]: Becoming Leader. State: Replica: 2a34390249924f8596c5f181e7a2ef2e, State: Running, Role: LEADER
I20260620 23:06:07.966768 30471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } }
I20260620 23:06:07.966809 30466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:07.968863 30466 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 23:06:07.968845 30473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a34390249924f8596c5f181e7a2ef2e. Latest consensus state: current_term: 1 leader_uuid: "2a34390249924f8596c5f181e7a2ef2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } } }
I20260620 23:06:07.968935 30473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:07.969157 30486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:06:07.969410 30472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a34390249924f8596c5f181e7a2ef2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } } }
I20260620 23:06:07.969524 30472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:07.969647 30486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:06:07 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:07.976431 30470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:07.962464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.17.188.126:34301)
0620 23:06:07.962755 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:07.962764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:07.962789 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:06:07.963346 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:07.963355 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:07.963413 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:07.963555 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:07.964303 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:07.965798 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:07.966873 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:07.967519 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:07.967625 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:07.971929 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:07.971979 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:07.971979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:07.972068 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:07.974718 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:07.975155 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:07.975158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:07.975158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:07.975247 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:07.975478 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:07.975479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:07.975479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:07.975780 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:07.975932 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:07.976079 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 23:06:07.976221 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":60,"threads_started":1}
I20260620 23:06:07.977741 30486 catalog_manager.cc:1347] Generated new cluster ID: 3b73e57a46a24ccdb7bf60a363d85346
I20260620 23:06:07.977789 30486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:06:07.986635 30486 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:06:07.987411 30486 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:06:07.990993 30486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Generated new TSK 0
I20260620 23:06:07.991196 30486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:06:08.664280 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:35453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34301
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:08.778746 30511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:08.778990 30511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:08.779027 30511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:08.780758 30511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:08.780821 30511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:08.780861 30511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:06:08.783107 30511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.188.84:35453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:08.783351 30511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:08.783602 30511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:08.788704 30519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:08.788707 30516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:08.788784 30517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:08.788988 30511 server_base.cc:1034] running on GCE node
I20260620 23:06:08.789300 30511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:08.789616 30511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:08.790771 30511 hybrid_clock.cc:648] HybridClock initialized: now 1781996768790740 us; error 45 us; skew 500 ppm
Jun 20 23:06:08 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:08.793409 30511 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:06:08.793825 30511 webserver.cc:460] Webserver started at http://127.17.188.65:46227/ using document root <none> and password file <none>
I20260620 23:06:08.794050 30511 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:08.794114 30511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:08.794241 30511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:06:08 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:08.806188 30511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "2b82ce5388084ac6ae9b1b334db88b61"
format_stamp: "Formatted at 2026-06-20 23:06:08 on dist-test-slave-zpss"
server_key: "56a6fb5ef031f16cfd33222b9e352d06"
server_key_iv: "128277e56b4f769fa4fe3e8b0fb0b06f"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:08.806650 30511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "2b82ce5388084ac6ae9b1b334db88b61"
format_stamp: "Formatted at 2026-06-20 23:06:08 on dist-test-slave-zpss"
server_key: "56a6fb5ef031f16cfd33222b9e352d06"
server_key_iv: "128277e56b4f769fa4fe3e8b0fb0b06f"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:08.812757 30511 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260620 23:06:08.831350 30528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:08.831588 30511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:06:08.831667 30511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "2b82ce5388084ac6ae9b1b334db88b61"
format_stamp: "Formatted at 2026-06-20 23:06:08 on dist-test-slave-zpss"
server_key: "56a6fb5ef031f16cfd33222b9e352d06"
server_key_iv: "128277e56b4f769fa4fe3e8b0fb0b06f"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:08.831748 30511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:08.856500 30511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:08.858759 30511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:08.858929 30511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:08.859169 30511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:08.859586 30511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:08.859642 30511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:08.859686 30511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:08.859715 30511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:08.867452 30511 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:36345
I20260620 23:06:08.867560 30641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:36345 every 8 connection(s)
I20260620 23:06:08.867877 30511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:06:08 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:08.874963 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 30511
I20260620 23:06:08.875058 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:06:08.875270 18161 external_mini_cluster.cc:1428] Setting key 7c8cd174da1bdb46d7190801b41f072c
I20260620 23:06:08.885792 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:08.868700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34301 (local address 127.17.188.65:42369)
0620 23:06:08.869041 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:08.869073 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:08.869617 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:08.869961 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:08.869965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:08.870153 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:08.870334 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:08.870339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:08.871026 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:08.871027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:08.871809 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:08.871811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:08.872245 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:08.872246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:08.872373 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:08.872915 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:08.872923 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:08.874276 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:08.883732 (+ 9456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:08.883739 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:08.883742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:08.884050 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:08.884310 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:08.884311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:08.884311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:08.884442 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:08.884836 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:08.884842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:08.885004 (+ 162us) client_negotiation.cc:769] Sending connection context
0620 23:06:08.885125 (+ 121us) client_negotiation.cc:240] Negotiation successful
0620 23:06:08.885372 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":86,"threads_started":1}
I20260620 23:06:08.886585 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:08.868711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:42369 (local address 127.17.188.126:34301)
0620 23:06:08.869765 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:08.869767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:08.869773 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:06:08.869795 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:08.869795 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:08.869805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:08.869888 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:08.870438 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:08.870916 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:08.871924 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:08.872134 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:08.872222 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:08.881575 (+ 9353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:08.881577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:08.881577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:08.881586 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:08.883628 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:08.884147 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:08.884148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:08.884148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:08.884217 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:08.884522 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:08.884523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:08.884523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:08.884733 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:08.884805 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:08.886371 (+ 1566us) server_negotiation.cc:299] Negotiation successful
0620 23:06:08.886472 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1026,"thread_start_us":89,"threads_started":1}
I20260620 23:06:08.887138 30642 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34301
I20260620 23:06:08.887228 30642 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:08.887400 30642 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
I20260620 23:06:08.887984 30430 ts_manager.cc:194] Registered new tserver with Master: 2b82ce5388084ac6ae9b1b334db88b61 (127.17.188.65:36345)
I20260620 23:06:08.888823 30430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:42369
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:06:08.918166 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:35453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34301
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:06:08 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 23:06:09.030237 30650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:09.030494 30650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:09.030529 30650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:09.032177 30650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:09.032271 30650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:09.032339 30650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:06:09.034417 30650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.188.84:35453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:09.034646 30650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:09.034912 30650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:09.041923 30656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.041965 30655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.042439 30658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:09.042783 30650 server_base.cc:1034] running on GCE node
I20260620 23:06:09.042965 30650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:09.043272 30650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:09.044440 30650 hybrid_clock.cc:648] HybridClock initialized: now 1781996769044423 us; error 32 us; skew 500 ppm
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:09.047273 30650 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:06:09.047693 30650 webserver.cc:460] Webserver started at http://127.17.188.66:46273/ using document root <none> and password file <none>
I20260620 23:06:09.047938 30650 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:09.048009 30650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:09.048159 30650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:09.062291 30650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "510f1b64de6b4ada9c085150c621b528"
format_stamp: "Formatted at 2026-06-20 23:06:09 on dist-test-slave-zpss"
server_key: "4a3e77fdd2ff69afa632d8e841875aef"
server_key_iv: "3cfb776feac1114343e2bfa3d16a8603"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:09.062793 30650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "510f1b64de6b4ada9c085150c621b528"
format_stamp: "Formatted at 2026-06-20 23:06:09 on dist-test-slave-zpss"
server_key: "4a3e77fdd2ff69afa632d8e841875aef"
server_key_iv: "3cfb776feac1114343e2bfa3d16a8603"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:09.064755 30650 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 23:06:09.085848 30666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.093544 30650 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.000s sys 0.001s
I20260620 23:06:09.093668 30650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "510f1b64de6b4ada9c085150c621b528"
format_stamp: "Formatted at 2026-06-20 23:06:09 on dist-test-slave-zpss"
server_key: "4a3e77fdd2ff69afa632d8e841875aef"
server_key_iv: "3cfb776feac1114343e2bfa3d16a8603"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:09.093775 30650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:09.120239 30650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:09.122911 30650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:09.123126 30650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:09.123402 30650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:09.123828 30650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:09.123914 30650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.123978 30650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:09.124037 30650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.131013 30650 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:36473
I20260620 23:06:09.131486 30650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:06:09.131883 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 30650
I20260620 23:06:09.131992 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:06:09.132241 18161 external_mini_cluster.cc:1428] Setting key 60145dd7f8d543858c18f2c26bad70c5
I20260620 23:06:09.141940 30779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:36473 every 8 connection(s)
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:09.160216 30783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:09.142690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34301 (local address 127.17.188.66:54945)
0620 23:06:09.144313 (+ 1623us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:09.144368 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:09.145060 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:09.145444 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:09.145449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:09.145808 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:09.148226 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:09.148234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.151650 (+ 3416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.151651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:09.152927 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:09.152930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.153539 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.153540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:09.153680 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:09.154308 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:09.154318 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:09.156042 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:09.158381 (+ 2339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:09.158387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:09.158390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:09.158721 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:09.158976 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:09.158977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:09.158977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:09.159119 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:09.159465 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:09.159472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:09.159658 (+ 186us) client_negotiation.cc:769] Sending connection context
0620 23:06:09.159765 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 23:06:09.159959 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1577,"thread_start_us":75,"threads_started":1}
I20260620 23:06:09.160533 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:09.142941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:54945 (local address 127.17.188.126:34301)
0620 23:06:09.143002 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:09.143004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:09.144500 (+ 1496us) server_negotiation.cc:407] Connection header received
0620 23:06:09.145183 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:09.145184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:09.145200 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:09.145286 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:09.148641 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.149502 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.153037 (+ 3535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.153379 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.153460 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:09.156151 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:09.156152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:09.156153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:09.156161 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:09.158268 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:09.158826 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:09.158827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:09.158828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:09.158895 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:09.159210 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:09.159212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:09.159212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:09.159367 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:09.159427 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:09.160303 (+ 876us) server_negotiation.cc:299] Negotiation successful
0620 23:06:09.160424 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:06:09.161283 30780 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34301
I20260620 23:06:09.161410 30780 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:09.161653 30780 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
I20260620 23:06:09.162046 30430 ts_manager.cc:194] Registered new tserver with Master: 510f1b64de6b4ada9c085150c621b528 (127.17.188.66:36473)
I20260620 23:06:09.162523 30430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:54945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:06:09.176646 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:35453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34301
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:09.302182 30785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:09.302536 30785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:09.302634 30785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:09.304175 30785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:09.304292 30785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:09.304388 30785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:06:09.306510 30785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.188.84:35453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:34301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:09.307031 30785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:09.307337 30785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:09.313766 30791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.314174 30793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.315203 30790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:09.315485 30785 server_base.cc:1034] running on GCE node
I20260620 23:06:09.315708 30785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:09.315949 30785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:09.317126 30785 hybrid_clock.cc:648] HybridClock initialized: now 1781996769317102 us; error 39 us; skew 500 ppm
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:09.320678 30785 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:06:09.321183 30785 webserver.cc:460] Webserver started at http://127.17.188.67:39361/ using document root <none> and password file <none>
I20260620 23:06:09.321450 30785 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:09.321519 30785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:09.321652 30785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:09.339648 30785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "6d5368fe1744447797be09ad472a8bff"
format_stamp: "Formatted at 2026-06-20 23:06:09 on dist-test-slave-zpss"
server_key: "40dbdce2ca2a5b22f6b929dded42e38f"
server_key_iv: "194fc9b48d87460fe67dacc5a9bbc065"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:09.340240 30785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "6d5368fe1744447797be09ad472a8bff"
format_stamp: "Formatted at 2026-06-20 23:06:09 on dist-test-slave-zpss"
server_key: "40dbdce2ca2a5b22f6b929dded42e38f"
server_key_iv: "194fc9b48d87460fe67dacc5a9bbc065"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:09.342947 30785 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 23:06:09.369155 30800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.369562 30785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260620 23:06:09.369673 30785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "6d5368fe1744447797be09ad472a8bff"
format_stamp: "Formatted at 2026-06-20 23:06:09 on dist-test-slave-zpss"
server_key: "40dbdce2ca2a5b22f6b929dded42e38f"
server_key_iv: "194fc9b48d87460fe67dacc5a9bbc065"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:09.369783 30785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:09.392661 30785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:09.397239 30785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:09.397464 30785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:09.397847 30785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:09.399277 30785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:09.399364 30785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.399430 30785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:09.399492 30785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.406499 30785 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:33647
I20260620 23:06:09.406985 30785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:06:09.410588 30913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:33647 every 8 connection(s)
I20260620 23:06:09.412094 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 30785
I20260620 23:06:09.412201 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:06:09.412380 18161 external_mini_cluster.cc:1428] Setting key 6af1f6c8e0007108dc9303f7c768c9a5
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:09.420228 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:09.408185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:48631 (local address 127.17.188.126:34301)
0620 23:06:09.408237 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:09.408239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:09.409170 (+ 931us) server_negotiation.cc:407] Connection header received
0620 23:06:09.410171 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:09.410173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:09.410186 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:09.410222 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:09.411085 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.411866 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.414518 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.414822 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.414863 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:09.417242 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:09.417243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:09.417244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:09.417250 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:09.418852 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:09.419305 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:09.419307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:09.419307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:09.419362 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:09.419609 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:09.419610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:09.419610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:09.419722 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:09.419758 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:09.420062 (+ 304us) server_negotiation.cc:299] Negotiation successful
0620 23:06:09.420143 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:06:09.420984 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:09.408184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34301 (local address 127.17.188.67:48631)
0620 23:06:09.409042 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:09.409085 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:09.409999 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:09.410424 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:09.410428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:09.410697 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:09.410969 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:09.410975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.413141 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.413141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:09.414417 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:09.414420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.414955 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.414956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:09.415095 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:09.415674 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:09.415683 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:09.417148 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:09.418943 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:09.418951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:09.418953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:09.419228 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:09.419430 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:09.419431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:09.419431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:09.419543 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:09.419797 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:09.419802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:09.419936 (+ 134us) client_negotiation.cc:769] Sending connection context
0620 23:06:09.420621 (+ 685us) client_negotiation.cc:240] Negotiation successful
0620 23:06:09.420770 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"thread_start_us":558,"threads_started":1}
I20260620 23:06:09.421581 30914 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34301
I20260620 23:06:09.421661 30914 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:09.421825 30914 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
I20260620 23:06:09.422194 30430 ts_manager.cc:194] Registered new tserver with Master: 6d5368fe1744447797be09ad472a8bff (127.17.188.67:33647)
I20260620 23:06:09.422636 30430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:48631
I20260620 23:06:09.423856 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:06:09.424619 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 30511
I20260620 23:06:09.431746 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 30650
I20260620 23:06:09.436532 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 30785
I20260620 23:06:09.441700 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 30398
I20260620 23:06:09.448036 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34301
--webserver_interface=127.17.188.126
--webserver_port=42065
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:34301
--ranger_config_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:35453/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.188.84:35453/kms with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:09.545408 30917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:09.545626 30917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:09.545662 30917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:09.546773 30917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:09.546834 30917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:09.546857 30917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:09.546876 30917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:09.546905 30917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:09.548388 30917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.188.84:35453/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:34301
--ranger_config_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=42065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:09.548681 30917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:09.548920 30917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:09.553766 30925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.553746 30923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.553746 30922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:09.554198 30917 server_base.cc:1034] running on GCE node
I20260620 23:06:09.554457 30917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:09.554761 30917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:09.555931 30917 hybrid_clock.cc:648] HybridClock initialized: now 1781996769555916 us; error 30 us; skew 500 ppm
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:09.561528 30917 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:06:09.561939 30917 webserver.cc:460] Webserver started at http://127.17.188.126:42065/ using document root <none> and password file <none>
I20260620 23:06:09.562163 30917 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:09.562239 30917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:09.599159 30917 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:09.599885 30933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.600056 30917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:09.600140 30917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "2a34390249924f8596c5f181e7a2ef2e"
format_stamp: "Formatted at 2026-06-20 23:06:07 on dist-test-slave-zpss"
server_key: "88900b267cf869e3d9932784ca93118c"
server_key_iv: "54fac0d2a26db84f80fa100b64c2cfc9"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:09.600507 30917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:09.612314 30917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:09.614286 30917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:09.614452 30917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:09.619167 30917 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:34301
I20260620 23:06:09.619241 30985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:34301 every 8 connection(s)
I20260620 23:06:09.619654 30917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:06:09.620854 30986 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:06:09.621613 30986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Bootstrap starting.
I20260620 23:06:09.624822 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 30917
I20260620 23:06:09.625157 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:35453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:36345
--local_ip_for_outbound_sockets=127.17.188.65
--tserver_master_addrs=127.17.188.126:34301
--webserver_port=46227
--webserver_interface=127.17.188.65
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.188.84:35453/kms with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:09.630569 30986 log.cc:824] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:09.632783 30986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:06:09.633073 30986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Bootstrap complete.
I20260620 23:06:09.637053 30986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } }
I20260620 23:06:09.637388 30986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a34390249924f8596c5f181e7a2ef2e, State: Initialized, Role: FOLLOWER
I20260620 23:06:09.637593 30986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } }
I20260620 23:06:09.637658 30986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:06:09.637692 30986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:06:09.637758 30986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:06:09.639092 30986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } }
I20260620 23:06:09.639207 30986 leader_election.cc:302] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a34390249924f8596c5f181e7a2ef2e; no voters:
I20260620 23:06:09.639400 30986 leader_election.cc:288] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:06:09.639464 30990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:06:09.639732 30986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:09.639734 30990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [term 2 LEADER]: Becoming Leader. State: Replica: 2a34390249924f8596c5f181e7a2ef2e, State: Running, Role: LEADER
I20260620 23:06:09.641155 30990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } }
I20260620 23:06:09.641795 30986 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 23:06:09.643743 30991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2a34390249924f8596c5f181e7a2ef2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } } }
I20260620 23:06:09.644083 30991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:09.643743 30992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a34390249924f8596c5f181e7a2ef2e. Latest consensus state: current_term: 2 leader_uuid: "2a34390249924f8596c5f181e7a2ef2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a34390249924f8596c5f181e7a2ef2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34301 } } }
I20260620 23:06:09.644138 30992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:09.644510 31007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:06:09.645257 31007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:06:09.645483 31007 catalog_manager.cc:1259] Loaded cluster ID: 3b73e57a46a24ccdb7bf60a363d85346
I20260620 23:06:09.645534 31007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:06:09.646863 31007 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:06:09.647039 31007 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Loaded TSK: 0
I20260620 23:06:09.647245 31007 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:06:09.717265 30988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:09.717527 30988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:09.717563 30988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:09.718706 30988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:09.718811 30988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:09.718871 30988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:06:09.720629 30988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.188.84:35453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:36345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=46227
--tserver_master_addrs=127.17.188.126:34301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:09.720857 30988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:09.721081 30988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:09.725783 31023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.726387 31024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.726725 31026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:09.726938 30988 server_base.cc:1034] running on GCE node
I20260620 23:06:09.727124 30988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:09.727377 30988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:09.728523 30988 hybrid_clock.cc:648] HybridClock initialized: now 1781996769728505 us; error 35 us; skew 500 ppm
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:09.730947 30988 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:06:09.731447 30988 webserver.cc:460] Webserver started at http://127.17.188.65:46227/ using document root <none> and password file <none>
I20260620 23:06:09.731689 30988 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:09.731768 30988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:09.754936 30988 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:06:09.760594 31034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.760823 30988 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260620 23:06:09.760901 30988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "2b82ce5388084ac6ae9b1b334db88b61"
format_stamp: "Formatted at 2026-06-20 23:06:08 on dist-test-slave-zpss"
server_key: "56a6fb5ef031f16cfd33222b9e352d06"
server_key_iv: "128277e56b4f769fa4fe3e8b0fb0b06f"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:09.761363 30988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:09.772503 30988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:09.774880 30988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:09.775064 30988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:09.775285 30988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:09.775657 30988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:09.775743 30988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.775800 30988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:09.775841 30988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.782159 30988 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:36345
I20260620 23:06:09.782554 30988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:06:09.789822 31147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:36345 every 8 connection(s)
I20260620 23:06:09.791640 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 30988
I20260620 23:06:09.791945 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:35453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:36473
--local_ip_for_outbound_sockets=127.17.188.66
--tserver_master_addrs=127.17.188.126:34301
--webserver_port=46273
--webserver_interface=127.17.188.66
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.188.84:35453/kms with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:09.804015 31150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:09.783367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34301 (local address 127.17.188.65:44251)
0620 23:06:09.783553 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:09.783601 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:09.784218 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:09.791434 (+ 7216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:09.791438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:09.791656 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:09.791946 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:09.791952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.795070 (+ 3118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.795071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:09.796327 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:09.796329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.797149 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.797151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:09.797294 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:09.797910 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:09.797919 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:09.799479 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:09.802002 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:09.802007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:09.802009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:09.802318 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:09.802613 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:09.802614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:09.802614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:09.802748 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:09.803374 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:09.803379 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:09.803557 (+ 178us) client_negotiation.cc:769] Sending connection context
0620 23:06:09.803654 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 23:06:09.803796 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260620 23:06:09.806084 31151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:09.783420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:44251 (local address 127.17.188.126:34301)
0620 23:06:09.790938 (+ 7518us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:09.790945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:09.790974 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:06:09.791169 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:09.791176 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:09.791243 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:09.791397 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:09.793590 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.794941 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.796448 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.797031 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.797645 (+ 614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:09.799633 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:09.799637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:09.799637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:09.799648 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:09.801899 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:09.802415 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:09.802418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:09.802418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:09.802530 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:09.802826 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:09.802826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:09.802827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:09.803245 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:09.804926 (+ 1681us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:09.805265 (+ 339us) server_negotiation.cc:299] Negotiation successful
0620 23:06:09.805402 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7471,"thread_start_us":100,"threads_started":1}
I20260620 23:06:09.807269 31148 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34301
I20260620 23:06:09.807353 31148 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:09.807525 31148 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
W20260620 23:06:09.808351 31148 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:34301 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 23:06:09.808414 31148 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:09.808470 31148 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
I20260620 23:06:09.808842 31148 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:09.808907 31148 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
W20260620 23:06:09.809218 31148 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:06:09.914857 31152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:09.915122 31152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:09.915163 31152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:09.916837 31152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:09.916896 31152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:09.916940 31152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:06:09.918579 31152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.188.84:35453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:36473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=46273
--tserver_master_addrs=127.17.188.126:34301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:09.918844 31152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:09.919073 31152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:09.924453 31157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.924402 31160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:09.924446 31158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:09.929530 31152 server_base.cc:1034] running on GCE node
I20260620 23:06:09.929816 31152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:09.930100 31152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:09.931244 31152 hybrid_clock.cc:648] HybridClock initialized: now 1781996769931225 us; error 40 us; skew 500 ppm
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:09.934407 31152 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:06:09.934893 31152 webserver.cc:460] Webserver started at http://127.17.188.66:46273/ using document root <none> and password file <none>
I20260620 23:06:09.935210 31152 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:09.935314 31152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:09.957490 31152 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:09.958585 31168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.958794 31152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:09.958949 31152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "510f1b64de6b4ada9c085150c621b528"
format_stamp: "Formatted at 2026-06-20 23:06:09 on dist-test-slave-zpss"
server_key: "4a3e77fdd2ff69afa632d8e841875aef"
server_key_iv: "3cfb776feac1114343e2bfa3d16a8603"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:09.959482 31152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:09.977954 31152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:09.980089 31152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:09.980257 31152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:09.980512 31152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:09.980842 31152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:09.980887 31152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.980933 31152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:09.980950 31152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:09.988459 31152 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:36473
I20260620 23:06:09.988564 31281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:36473 every 8 connection(s)
I20260620 23:06:09.989018 31152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:06:09.993358 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 31152
I20260620 23:06:09.993649 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.188.84:35453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:33647
--local_ip_for_outbound_sockets=127.17.188.67
--tserver_master_addrs=127.17.188.126:34301
--webserver_port=39361
--webserver_interface=127.17.188.67
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.188.84:35453/kms with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:06:09 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:09.999718 31284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:09.989991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34301 (local address 127.17.188.66:47001)
0620 23:06:09.990198 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:09.990243 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:09.990899 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:09.991130 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:09.991135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:09.991357 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:09.991629 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:09.991636 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.992429 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.992430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:09.993619 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:09.993621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.994170 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.994171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:09.994303 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:09.994878 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:09.994888 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:09.996417 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:09.998218 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:09.998224 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:09.998226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:09.998505 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:09.998704 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:09.998705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:09.998705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:09.998821 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:09.999092 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:09.999099 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:09.999247 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 23:06:09.999362 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 23:06:09.999511 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":77,"threads_started":1}
I20260620 23:06:10.000168 31151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:09.990041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:47001 (local address 127.17.188.126:34301)
0620 23:06:09.990115 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:09.990117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:09.990341 (+ 224us) server_negotiation.cc:407] Connection header received
0620 23:06:09.991003 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:09.991003 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:09.991016 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:09.991058 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:09.991723 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.992328 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.993716 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:09.994028 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:09.994069 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:09.996510 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:09.996511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:09.996512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:09.996519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:09.998133 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:09.998585 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:09.998586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:09.998587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:09.998642 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:09.998886 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:09.998887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:09.998887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:09.999003 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:09.999065 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:10.000001 (+ 936us) server_negotiation.cc:299] Negotiation successful
0620 23:06:10.000084 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:06:10.000927 31282 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34301
I20260620 23:06:10.001075 31282 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:10.001304 31282 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
W20260620 23:06:10.002034 31282 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:34301 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 23:06:10.002162 31282 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:10.002267 31282 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
I20260620 23:06:10.002671 31282 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:10.002785 31282 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
W20260620 23:06:10.003137 31282 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:06:10.092360 31285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:10.092624 31285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:10.092658 31285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:10.094336 31285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:10.094460 31285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:10.094542 31285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:06:10.096699 31285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.188.84:35453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:33647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=39361
--tserver_master_addrs=127.17.188.126:34301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:10.096933 31285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:10.097275 31285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:10.103358 31290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:10.103379 31293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:10.103358 31291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:10.104298 31285 server_base.cc:1034] running on GCE node
I20260620 23:06:10.104511 31285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:10.104782 31285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:10.105962 31285 hybrid_clock.cc:648] HybridClock initialized: now 1781996770105944 us; error 31 us; skew 500 ppm
Jun 20 23:06:10 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996770, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:10.109141 31285 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:06:10.109720 31285 webserver.cc:460] Webserver started at http://127.17.188.67:39361/ using document root <none> and password file <none>
I20260620 23:06:10.110021 31285 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:10.110111 31285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:06:10 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996770, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:10.129729 31285 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260620 23:06:10.133734 31301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:10.133925 31285 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260620 23:06:10.133997 31285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "6d5368fe1744447797be09ad472a8bff"
format_stamp: "Formatted at 2026-06-20 23:06:09 on dist-test-slave-zpss"
server_key: "40dbdce2ca2a5b22f6b929dded42e38f"
server_key_iv: "194fc9b48d87460fe67dacc5a9bbc065"
server_key_version: "kuduclusterkey@0"
I20260620 23:06:10.134444 31285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:10.154819 31285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:10.157167 31285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:10.157379 31285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:10.157662 31285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:10.158047 31285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:10.158118 31285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:10.158193 31285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:10.158239 31285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:10.165938 31285 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:33647
I20260620 23:06:10.166054 31415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:33647 every 8 connection(s)
I20260620 23:06:10.166471 31285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:06:10.170629 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 31285
Jun 20 23:06:10 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996770, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:10.177678 31151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:10.167273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:52639 (local address 127.17.188.126:34301)
0620 23:06:10.167320 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:10.167322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:10.167565 (+ 243us) server_negotiation.cc:407] Connection header received
0620 23:06:10.168251 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:10.168253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:10.168265 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:10.168295 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:10.169064 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:10.169873 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:10.171573 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:10.171841 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:10.171889 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:10.174463 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:10.174464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:10.174464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:10.174472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:10.176159 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:10.176610 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:10.176611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:10.176612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:10.176667 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:10.176933 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:10.176934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:10.176934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:10.177053 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:10.177140 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:10.177511 (+ 371us) server_negotiation.cc:299] Negotiation successful
0620 23:06:10.177596 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:06:10.177901 31418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:10.167279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34301 (local address 127.17.188.67:52639)
0620 23:06:10.167434 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:10.167478 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:10.168096 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:10.168467 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:10.168472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:10.168687 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:10.168939 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:10.168944 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:10.170001 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:10.170002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:10.171475 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:10.171478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:10.172022 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:10.172023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:10.172156 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:10.172725 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:10.172734 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:10.174372 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:10.176249 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:10.176255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:10.176257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:10.176526 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:10.176745 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:10.176745 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:10.176746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:10.176861 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:10.177131 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:10.177137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:10.177305 (+ 168us) client_negotiation.cc:769] Sending connection context
0620 23:06:10.177475 (+ 170us) client_negotiation.cc:240] Negotiation successful
0620 23:06:10.177614 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260620 23:06:10.178375 31416 heartbeater.cc:347] Connected to a master server at 127.17.188.126:34301
I20260620 23:06:10.178468 31416 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:10.178640 31416 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
W20260620 23:06:10.179181 31416 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:34301 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 23:06:10.179243 31416 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:10.179303 31416 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
I20260620 23:06:10.179610 31416 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:10.179680 31416 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
W20260620 23:06:10.180018 31416 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:06:10.181650 31419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:10.171018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55220 (local address 127.17.188.126:34301)
0620 23:06:10.172414 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:10.172416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:10.172553 (+ 137us) server_negotiation.cc:407] Connection header received
0620 23:06:10.172710 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:10.172712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:10.172724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:10.172797 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:10.174195 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:10.175640 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:10.176765 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:10.177067 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:10.177140 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:10.178569 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:10.178570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:10.178570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:10.178577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:10.180213 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:10.180593 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:10.180594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:10.180594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:10.180641 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:10.180843 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:10.180843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:10.180844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:10.180963 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:10.181150 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:10.181464 (+ 314us) server_negotiation.cc:299] Negotiation successful
0620 23:06:10.181549 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376,"thread_start_us":52,"threads_started":1}
Jun 20 23:06:10 dist-test-slave-zpss krb5kdc[29954](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996770, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:10.809406 31148 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:10.809561 31148 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
I20260620 23:06:10.809923 30950 ts_manager.cc:194] Registered new tserver with Master: 2b82ce5388084ac6ae9b1b334db88b61 (127.17.188.65:36345)
I20260620 23:06:10.810899 30950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:44251
I20260620 23:06:11.003364 31282 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:11.003538 31282 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
I20260620 23:06:11.003839 30950 ts_manager.cc:194] Registered new tserver with Master: 510f1b64de6b4ada9c085150c621b528 (127.17.188.66:36473)
I20260620 23:06:11.004271 30950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:47001
Jun 20 23:06:11 dist-test-slave-zpss krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996770, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for HTTP/127.17.188.84@KRBTEST.COM
I20260620 23:06:11.180187 31416 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:11.180322 31416 heartbeater.cc:511] Master 127.17.188.126:34301 requested a full tablet report, sending...
I20260620 23:06:11.180657 30950 ts_manager.cc:194] Registered new tserver with Master: 6d5368fe1744447797be09ad472a8bff (127.17.188.67:33647)
I20260620 23:06:11.181183 30950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:52639
I20260620 23:06:11.184806 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:06:11.197389 31438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.185805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.17.188.126:34301)
0620 23:06:11.189460 (+ 3655us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:11.189462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:11.189573 (+ 111us) server_negotiation.cc:407] Connection header received
0620 23:06:11.189635 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:11.189636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:11.189648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:11.189730 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:11.190482 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.191229 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.192190 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.192505 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.192567 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.194177 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:11.194179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:11.194179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:11.194187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:11.196088 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:11.196535 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:11.196536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:11.196536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:11.196606 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:11.196817 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:11.196818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:11.196818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:11.196951 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:11.197051 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:11.197165 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 23:06:11.197249 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3632,"thread_start_us":68,"threads_started":1}
I20260620 23:06:11.199589 30950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:06:11.201750 30950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:06:11.210580 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.205411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:36473 (local address 127.0.0.1:54874)
0620 23:06:11.206148 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:11.206158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:11.206209 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:11.206557 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:11.206560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:11.206562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:11.206663 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.206666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.207934 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.207935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.209194 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.209197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.210272 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.210273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.210365 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.210379 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 23:06:11.210450 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 23:06:11.210497 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":2,"thread_start_us":65,"threads_started":1}
I20260620 23:06:11.210934 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.205760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44632 (local address 127.17.188.65:36345)
0620 23:06:11.205913 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:11.205918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:11.206382 (+ 464us) server_negotiation.cc:407] Connection header received
0620 23:06:11.206419 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:11.206421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:11.206436 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:11.206517 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:11.206738 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.208103 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.209406 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.210394 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.210462 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.210510 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:11.210816 (+ 306us) server_negotiation.cc:299] Negotiation successful
0620 23:06:11.210867 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260620 23:06:11.211124 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.205537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.17.188.66:36473)
0620 23:06:11.205699 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:11.205704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:11.206243 (+ 539us) server_negotiation.cc:407] Connection header received
0620 23:06:11.206290 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:11.206293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:11.206310 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:11.206425 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:11.206763 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.207821 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.209310 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.210106 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.210183 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.210235 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:11.211003 (+ 768us) server_negotiation.cc:299] Negotiation successful
0620 23:06:11.211044 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260620 23:06:11.210754 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.205549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36345 (local address 127.0.0.1:44632)
0620 23:06:11.206278 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:11.206286 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:11.206320 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:11.206589 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:11.206590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:11.206592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:11.206668 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.206669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.208179 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.208180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.209284 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.209286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.210542 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.210543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.210602 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.210606 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:06:11.210649 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 23:06:11.210683 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":27,"thread_start_us":61,"threads_started":1}
I20260620 23:06:11.212049 31216 tablet_service.cc:1467] Processing CreateTablet for tablet 153785d1b3fb4ab2a9300f52280f1817 (DEFAULT_TABLE table=test-table [id=b2786fb9fc5043eb80301df7e67f83d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:11.212056 31082 tablet_service.cc:1467] Processing CreateTablet for tablet 153785d1b3fb4ab2a9300f52280f1817 (DEFAULT_TABLE table=test-table [id=b2786fb9fc5043eb80301df7e67f83d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:11.212401 31082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153785d1b3fb4ab2a9300f52280f1817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:11.212414 31216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153785d1b3fb4ab2a9300f52280f1817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:11.215893 31451 tablet_bootstrap.cc:492] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61: Bootstrap starting.
I20260620 23:06:11.216563 31451 tablet_bootstrap.cc:654] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:11.216867 31451 log.cc:824] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:11.217190 31450 tablet_bootstrap.cc:492] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528: Bootstrap starting.
I20260620 23:06:11.217703 31451 tablet_bootstrap.cc:492] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61: No bootstrap required, opened a new log
I20260620 23:06:11.217828 31451 ts_tablet_manager.cc:1397] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:06:11.219769 31444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.204942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33647 (local address 127.0.0.1:55130)
0620 23:06:11.205130 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:11.205176 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:11.205250 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:11.216221 (+ 10971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:11.216222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:11.216224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:11.216264 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.216266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.217448 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.217449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.218527 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.218530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.219548 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.219549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.219616 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.219621 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:06:11.219657 (+ 36us) client_negotiation.cc:240] Negotiation successful
0620 23:06:11.219693 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
I20260620 23:06:11.220207 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.205198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55130 (local address 127.17.188.67:33647)
0620 23:06:11.216026 (+ 10828us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:11.216031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:11.216043 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:06:11.216090 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:11.216092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:11.216109 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:11.216344 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:11.216436 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.217279 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.218645 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.219431 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.219981 (+ 550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.220054 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:11.220085 (+ 31us) server_negotiation.cc:299] Negotiation successful
0620 23:06:11.220127 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10790,"thread_start_us":52,"threads_started":1}
I20260620 23:06:11.220182 31451 raft_consensus.cc:348] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } }
I20260620 23:06:11.220417 31451 raft_consensus.cc:374] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:11.220453 31451 raft_consensus.cc:729] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b82ce5388084ac6ae9b1b334db88b61, State: Initialized, Role: FOLLOWER
I20260620 23:06:11.220575 31451 consensus_queue.cc:260] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } }
I20260620 23:06:11.221148 31451 ts_tablet_manager.cc:1428] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:06:11.221264 31148 heartbeater.cc:503] Master 127.17.188.126:34301 was elected leader, sending a full tablet report...
I20260620 23:06:11.221493 31347 tablet_service.cc:1467] Processing CreateTablet for tablet 153785d1b3fb4ab2a9300f52280f1817 (DEFAULT_TABLE table=test-table [id=b2786fb9fc5043eb80301df7e67f83d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:11.221835 31347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153785d1b3fb4ab2a9300f52280f1817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:11.222530 31450 tablet_bootstrap.cc:654] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:11.222827 31450 log.cc:824] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:11.225925 31450 tablet_bootstrap.cc:492] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528: No bootstrap required, opened a new log
I20260620 23:06:11.225994 31450 ts_tablet_manager.cc:1397] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260620 23:06:11.228523 31450 raft_consensus.cc:348] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } }
I20260620 23:06:11.228678 31450 raft_consensus.cc:374] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:11.228709 31450 raft_consensus.cc:729] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 510f1b64de6b4ada9c085150c621b528, State: Initialized, Role: FOLLOWER
I20260620 23:06:11.228904 31455 tablet_bootstrap.cc:492] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff: Bootstrap starting.
I20260620 23:06:11.228984 31450 consensus_queue.cc:260] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } }
I20260620 23:06:11.229234 31450 ts_tablet_manager.cc:1428] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:06:11.229394 31282 heartbeater.cc:503] Master 127.17.188.126:34301 was elected leader, sending a full tablet report...
I20260620 23:06:11.229727 31455 tablet_bootstrap.cc:654] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:11.230053 31455 log.cc:824] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:11.230855 31455 tablet_bootstrap.cc:492] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff: No bootstrap required, opened a new log
I20260620 23:06:11.230921 31455 ts_tablet_manager.cc:1397] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:06:11.232754 31455 raft_consensus.cc:348] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } }
I20260620 23:06:11.232910 31455 raft_consensus.cc:374] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:11.232944 31455 raft_consensus.cc:729] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5368fe1744447797be09ad472a8bff, State: Initialized, Role: FOLLOWER
I20260620 23:06:11.233038 31455 consensus_queue.cc:260] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } }
I20260620 23:06:11.233256 31455 ts_tablet_manager.cc:1428] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 23:06:11.233305 31416 heartbeater.cc:503] Master 127.17.188.126:34301 was elected leader, sending a full tablet report...
W20260620 23:06:11.240156 31283 tablet.cc:2369] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:06:11.268792 31456 raft_consensus.cc:482] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:11.268934 31456 raft_consensus.cc:504] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } }
I20260620 23:06:11.269631 31456 leader_election.cc:288] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b82ce5388084ac6ae9b1b334db88b61 (127.17.188.65:36345), 6d5368fe1744447797be09ad472a8bff (127.17.188.67:33647)
I20260620 23:06:11.273283 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.269571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:40045 (local address 127.17.188.67:33647)
0620 23:06:11.269611 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:11.269613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:11.270055 (+ 442us) server_negotiation.cc:407] Connection header received
0620 23:06:11.270146 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:11.270147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:11.270155 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:11.270182 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:11.270371 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.271085 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.272377 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.272914 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.272961 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.272969 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:11.273190 (+ 221us) server_negotiation.cc:299] Negotiation successful
0620 23:06:11.273224 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:06:11.273532 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.269403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33647 (local address 127.17.188.66:40045)
0620 23:06:11.269955 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:11.270010 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:11.270081 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:11.270237 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:11.270237 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:11.270239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:11.270307 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.270309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.271172 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.271173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.272281 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.272284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.273045 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.273046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.273107 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.273110 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:06:11.273405 (+ 295us) client_negotiation.cc:240] Negotiation successful
0620 23:06:11.273446 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":5,"thread_start_us":81,"threads_started":1}
I20260620 23:06:11.273679 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.269947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:48013 (local address 127.17.188.65:36345)
0620 23:06:11.269993 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:11.269995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:11.270255 (+ 260us) server_negotiation.cc:407] Connection header received
0620 23:06:11.270397 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:11.270398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:11.270409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:11.270444 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:11.270689 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.271391 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.272662 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.273229 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.273282 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.273291 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:11.273569 (+ 278us) server_negotiation.cc:299] Negotiation successful
0620 23:06:11.273611 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:06:11.273799 31358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153785d1b3fb4ab2a9300f52280f1817" candidate_uuid: "510f1b64de6b4ada9c085150c621b528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5368fe1744447797be09ad472a8bff" is_pre_election: true
I20260620 23:06:11.273911 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.269813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36345 (local address 127.17.188.66:48013)
0620 23:06:11.270177 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:11.270190 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:11.270287 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:11.270517 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:11.270518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:11.270520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:11.270597 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.270599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.271501 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.271502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.272553 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:11.272556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.273396 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.273397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:11.273475 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.273479 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:06:11.273797 (+ 318us) client_negotiation.cc:240] Negotiation successful
0620 23:06:11.273838 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":85,"threads_started":1}
I20260620 23:06:11.273933 31358 raft_consensus.cc:2454] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 510f1b64de6b4ada9c085150c621b528 in term 0.
I20260620 23:06:11.274143 31172 leader_election.cc:302] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 510f1b64de6b4ada9c085150c621b528, 6d5368fe1744447797be09ad472a8bff; no voters:
I20260620 23:06:11.274245 31456 raft_consensus.cc:2781] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:06:11.274276 31456 raft_consensus.cc:482] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:06:11.274294 31456 raft_consensus.cc:3037] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:11.274266 31102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153785d1b3fb4ab2a9300f52280f1817" candidate_uuid: "510f1b64de6b4ada9c085150c621b528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b82ce5388084ac6ae9b1b334db88b61" is_pre_election: true
I20260620 23:06:11.274434 31102 raft_consensus.cc:2454] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 510f1b64de6b4ada9c085150c621b528 in term 0.
I20260620 23:06:11.275068 31456 raft_consensus.cc:504] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } }
I20260620 23:06:11.275177 31456 leader_election.cc:288] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [CANDIDATE]: Term 1 election: Requested vote from peers 2b82ce5388084ac6ae9b1b334db88b61 (127.17.188.65:36345), 6d5368fe1744447797be09ad472a8bff (127.17.188.67:33647)
I20260620 23:06:11.275334 31358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153785d1b3fb4ab2a9300f52280f1817" candidate_uuid: "510f1b64de6b4ada9c085150c621b528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5368fe1744447797be09ad472a8bff"
I20260620 23:06:11.275400 31358 raft_consensus.cc:3037] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:11.275465 31102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153785d1b3fb4ab2a9300f52280f1817" candidate_uuid: "510f1b64de6b4ada9c085150c621b528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b82ce5388084ac6ae9b1b334db88b61"
I20260620 23:06:11.275543 31102 raft_consensus.cc:3037] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:11.276161 31358 raft_consensus.cc:2454] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 510f1b64de6b4ada9c085150c621b528 in term 1.
I20260620 23:06:11.276321 31172 leader_election.cc:302] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 510f1b64de6b4ada9c085150c621b528, 6d5368fe1744447797be09ad472a8bff; no voters:
I20260620 23:06:11.276391 31456 raft_consensus.cc:2781] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:11.276464 31102 raft_consensus.cc:2454] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 510f1b64de6b4ada9c085150c621b528 in term 1.
I20260620 23:06:11.276497 31456 raft_consensus.cc:686] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 1 LEADER]: Becoming Leader. State: Replica: 510f1b64de6b4ada9c085150c621b528, State: Running, Role: LEADER
I20260620 23:06:11.276569 31456 consensus_queue.cc:237] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } }
I20260620 23:06:11.277282 30950 catalog_manager.cc:5495] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 reported cstate change: term changed from 0 to 1, leader changed from <none> to 510f1b64de6b4ada9c085150c621b528 (127.17.188.66). New cstate: current_term: 1 leader_uuid: "510f1b64de6b4ada9c085150c621b528" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510f1b64de6b4ada9c085150c621b528" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 36473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 } health_report { overall_health: UNKNOWN } } }
W20260620 23:06:11.291585 31149 tablet.cc:2369] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:06:11.301347 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.298056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54886 (local address 127.17.188.66:36473)
0620 23:06:11.298100 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:11.298102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:11.298141 (+ 39us) server_negotiation.cc:407] Connection header received
0620 23:06:11.298241 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:11.298242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:11.298261 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:11.298325 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:06:11.298520 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.299228 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.300381 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:11.300667 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:11.300755 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:11.300950 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:06:11.301074 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:06:11.301151 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:11.301234 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 23:06:11.301254 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:06:11.303318 31102 raft_consensus.cc:1261] T 153785d1b3fb4ab2a9300f52280f1817 P 2b82ce5388084ac6ae9b1b334db88b61 [term 1 FOLLOWER]: Refusing update from remote peer 510f1b64de6b4ada9c085150c621b528: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:11.303591 31358 raft_consensus.cc:1261] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [term 1 FOLLOWER]: Refusing update from remote peer 510f1b64de6b4ada9c085150c621b528: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:11.303718 31461 consensus_queue.cc:1035] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b82ce5388084ac6ae9b1b334db88b61" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:11.303936 31461 consensus_queue.cc:1035] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5368fe1744447797be09ad472a8bff" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:11.307287 31464 mvcc.cc:205] Tried to move back new op lower bound from 7299058775255994368 to 7299058775149461504. Current Snapshot: MvccSnapshot[applied={T|T < 7299058775255994368}]
I20260620 23:06:11.307787 31465 mvcc.cc:205] Tried to move back new op lower bound from 7299058775255994368 to 7299058775149461504. Current Snapshot: MvccSnapshot[applied={T|T < 7299058775255994368}]
I20260620 23:06:11.308545 31466 mvcc.cc:205] Tried to move back new op lower bound from 7299058775255994368 to 7299058775149461504. Current Snapshot: MvccSnapshot[applied={T|T < 7299058775255994368}]
I20260620 23:06:11.311486 30950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:11.311592 30950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:11.312350 30950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a34390249924f8596c5f181e7a2ef2e: Sending DeleteTablet for 3 replicas of tablet 153785d1b3fb4ab2a9300f52280f1817
I20260620 23:06:11.312724 31347 tablet_service.cc:1514] Processing DeleteTablet for tablet 153785d1b3fb4ab2a9300f52280f1817 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:11 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:55130
I20260620 23:06:11.312881 31082 tablet_service.cc:1514] Processing DeleteTablet for tablet 153785d1b3fb4ab2a9300f52280f1817 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:11 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:44632
I20260620 23:06:11.312954 31216 tablet_service.cc:1514] Processing DeleteTablet for tablet 153785d1b3fb4ab2a9300f52280f1817 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:11 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:54874
I20260620 23:06:11.313378 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 30988
I20260620 23:06:11.323035 31473 tablet_replica.cc:330] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff: stopping tablet replica
I20260620 23:06:11.323144 31473 raft_consensus.cc:2229] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:06:11.323273 31473 raft_consensus.cc:2258] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:11.323468 31475 tablet_replica.cc:330] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528: stopping tablet replica
I20260620 23:06:11.323570 31475 raft_consensus.cc:2229] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:06:11.323696 31475 raft_consensus.cc:2258] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 23:06:11.324728 30936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:36345 (error 108)
I20260620 23:06:11.325193 31475 ts_tablet_manager.cc:1905] T 153785d1b3fb4ab2a9300f52280f1817 P 510f1b64de6b4ada9c085150c621b528: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:11.325591 31444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.325480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36345 (local address 127.0.0.1:44640)
0620 23:06:11.325526 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:11.325559 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 23:06:11.325699 30936 catalog_manager.cc:4584] TS 2b82ce5388084ac6ae9b1b334db88b61 (127.17.188.65:36345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 153785d1b3fb4ab2a9300f52280f1817: Network error: Client connection negotiation failed: client connection to 127.17.188.65:36345: connect: Connection refused (error 111)
I20260620 23:06:11.325875 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 31152
I20260620 23:06:11.331313 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 31285
I20260620 23:06:11.331398 31473 ts_tablet_manager.cc:1905] T 153785d1b3fb4ab2a9300f52280f1817 P 6d5368fe1744447797be09ad472a8bff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:11.331501 31444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:11.331409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:36473 (local address 127.0.0.1:54890)
0620 23:06:11.331451 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:11.331474 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.66:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 23:06:11.336645 31444 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:06:11.336437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33647 (local address 127.0.0.1:55132)
0620 23:06:11.336471 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:11.336480 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:11.336511 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:11.336607 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.67:33647: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":21}
I20260620 23:06:11.337083 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 30917
I20260620 23:06:11.342344 18161 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 23:06:11.676324 18161 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 23:06:11.676437 18161 mini_ranger.cc:66] Stopping Ranger...
I20260620 23:06:12.020143 18161 mini_ranger.cc:68] Stopped Ranger
2026-06-20 23:06:12.020 UTC [29984] LOG: received smart shutdown request
2026-06-20 23:06:12.023 UTC [29984] LOG: background worker "logical replication launcher" (PID 29993) exited with exit code 1
2026-06-20 23:06:12.023 UTC [29988] LOG: shutting down
2026-06-20 23:06:12.158 UTC [29984] LOG: database system is shut down
2026-06-20T23:06:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46147 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:06:12 dist-test-slave-zpss krb5kdc[31486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:06:12 dist-test-slave-zpss krb5kdc[31486](info): set up 2 sockets
Jun 20 23:06:12 dist-test-slave-zpss krb5kdc[31486](info): commencing operation
krb5kdc: starting...
W20260620 23:06:13.273036 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:06:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:06:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:06:13.378083 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45543
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:45543,127.17.188.125:45009,127.17.188.124:37183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:13.454212 31499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:13.454406 31499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:13.454452 31499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:13.455567 31499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:13.455608 31499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:13.455622 31499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:13.455647 31499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:13.455659 31499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:13.457926 31499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:45543,127.17.188.125:45009,127.17.188.124:37183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:13.458173 31499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:13.458420 31499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:13.463451 31507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:13.463654 31499 server_base.cc:1034] running on GCE node
W20260620 23:06:13.463373 31504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:13.463369 31505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:13.463995 31499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:13.464329 31499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:13.465482 31499 hybrid_clock.cc:648] HybridClock initialized: now 1781996773465449 us; error 43 us; skew 500 ppm
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:13.468293 31499 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:06:13.468770 31499 webserver.cc:460] Webserver started at http://127.17.188.126:38047/ using document root <none> and password file <none>
I20260620 23:06:13.468988 31499 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:13.469053 31499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:13.469177 31499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:13.470206 31499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "490b32cf099f44e584c24a02209397a5"
format_stamp: "Formatted at 2026-06-20 23:06:13 on dist-test-slave-zpss"
server_key: "b98921655fac5bc7ab01d600b01abc2c"
server_key_iv: "c5bfff8f64ad0388f5965fc34c9d1750"
server_key_version: "encryptionkey@0"
I20260620 23:06:13.470573 31499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "490b32cf099f44e584c24a02209397a5"
format_stamp: "Formatted at 2026-06-20 23:06:13 on dist-test-slave-zpss"
server_key: "b98921655fac5bc7ab01d600b01abc2c"
server_key_iv: "c5bfff8f64ad0388f5965fc34c9d1750"
server_key_version: "encryptionkey@0"
I20260620 23:06:13.471935 31499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:06:13.472596 31514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:13.472760 31499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:13.472864 31499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "490b32cf099f44e584c24a02209397a5"
format_stamp: "Formatted at 2026-06-20 23:06:13 on dist-test-slave-zpss"
server_key: "b98921655fac5bc7ab01d600b01abc2c"
server_key_iv: "c5bfff8f64ad0388f5965fc34c9d1750"
server_key_version: "encryptionkey@0"
I20260620 23:06:13.472940 31499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:13.504173 31499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:13.506656 31499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:13.506820 31499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:13.510924 31499 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:45543
I20260620 23:06:13.510962 31566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:45543 every 8 connection(s)
I20260620 23:06:13.511327 31499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:06:13.512045 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 31499
I20260620 23:06:13.512151 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:06:13.512171 31567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:13.512351 18161 external_mini_cluster.cc:1428] Setting key 93a30b4f758671ed812bfc2a9a309606
I20260620 23:06:13.513761 31567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.188.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.125@KRBTEST.COM" created.
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:13.526690 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.514401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45543 (local address 127.0.0.1:55506)
0620 23:06:13.514625 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.514667 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:13.515360 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:13.515676 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:13.515679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:13.516074 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:13.516303 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.516323 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.517315 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.517338 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.518474 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.518477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.518953 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.518955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.519041 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.519485 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:13.519494 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:13.523127 (+ 3633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:13.525246 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.525252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.525253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.525498 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.525757 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.525758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.525758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.525837 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.526300 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:13.526306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:13.526358 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 23:06:13.526425 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 23:06:13.526539 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260620 23:06:13.526796 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.514480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.17.188.126:45543)
0620 23:06:13.514639 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.514648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.514781 (+ 133us) server_negotiation.cc:407] Connection header received
0620 23:06:13.515521 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.515522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.515539 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.515635 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.516433 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.517215 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.518556 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.518863 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.518944 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.523241 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.523248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.523248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.523253 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.525162 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.525575 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.525578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.525579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.525685 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.525916 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.525917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.525918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.526227 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.526315 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.526503 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.526625 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260620 23:06:13.527633 31567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:06:13.527980 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.527847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:45009 (local address 127.0.0.1:51946)
0620 23:06:13.527897 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.527950 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 23:06:13.528120 31516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
I20260620 23:06:13.528541 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.528442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:45009 (local address 127.0.0.1:51958)
0620 23:06:13.528487 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.528519 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:06:13.528685 31567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.125:45009: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab.
Entry for principal kudu/127.17.188.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab.
I20260620 23:06:13.535800 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.125:45009
--webserver_interface=127.17.188.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab
--principal=kudu/127.17.188.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:45543,127.17.188.125:45009,127.17.188.124:37183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:13.577884 31567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } attempt: 1
I20260620 23:06:13.578313 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.578169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:45009 (local address 127.0.0.1:51962)
0620 23:06:13.578233 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.578276 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 23:06:13.578675 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.578583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:45009 (local address 127.0.0.1:51972)
0620 23:06:13.578627 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.578656 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:06:13.578796 31567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.125:45009: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
W20260620 23:06:13.613473 31574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:13.613646 31574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:13.613667 31574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:13.614825 31574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:13.614873 31574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:13.614887 31574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:13.614900 31574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:13.614912 31574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:13.616362 31574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:45543,127.17.188.125:45009,127.17.188.124:37183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab
--principal=kudu/127.17.188.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.125:45009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.188.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:13.616587 31574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:13.616793 31574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:06:13.621500 31574 server_base.cc:1034] running on GCE node
W20260620 23:06:13.621470 31580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:13.621470 31579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:13.621465 31582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:13.621861 31574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:13.622138 31574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:13.623271 31574 hybrid_clock.cc:648] HybridClock initialized: now 1781996773623263 us; error 32 us; skew 500 ppm
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:13.625563 31574 init.cc:377] Logged in from keytab as kudu/127.17.188.125@KRBTEST.COM (short username kudu)
I20260620 23:06:13.625959 31574 webserver.cc:460] Webserver started at http://127.17.188.125:41017/ using document root <none> and password file <none>
I20260620 23:06:13.626168 31574 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:13.626209 31574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:13.626353 31574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:13.627486 31574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/data/instance:
uuid: "a5a9bccea26e40eca7b9d488d6ea9edf"
format_stamp: "Formatted at 2026-06-20 23:06:13 on dist-test-slave-zpss"
server_key: "8b48e9023c953960d1abebf6a59788d0"
server_key_iv: "1581a848189ab66892693514a1ccf0f9"
server_key_version: "encryptionkey@0"
I20260620 23:06:13.627831 31574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/wal/instance:
uuid: "a5a9bccea26e40eca7b9d488d6ea9edf"
format_stamp: "Formatted at 2026-06-20 23:06:13 on dist-test-slave-zpss"
server_key: "8b48e9023c953960d1abebf6a59788d0"
server_key_iv: "1581a848189ab66892693514a1ccf0f9"
server_key_version: "encryptionkey@0"
I20260620 23:06:13.629101 31574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:06:13.629870 31589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:13.630054 31574 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:13.630143 31574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/wal
uuid: "a5a9bccea26e40eca7b9d488d6ea9edf"
format_stamp: "Formatted at 2026-06-20 23:06:13 on dist-test-slave-zpss"
server_key: "8b48e9023c953960d1abebf6a59788d0"
server_key_iv: "1581a848189ab66892693514a1ccf0f9"
server_key_version: "encryptionkey@0"
I20260620 23:06:13.630221 31574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:13.642382 31574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:13.644172 31574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:13.644323 31574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:13.646960 31567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } attempt: 2
I20260620 23:06:13.647336 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.647217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:45009 (local address 127.0.0.1:51986)
0620 23:06:13.647274 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.647307 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 23:06:13.647688 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.647619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:45009 (local address 127.0.0.1:51990)
0620 23:06:13.647647 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.647672 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260620 23:06:13.647792 31567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.125:45009: Network error: Client connection negotiation failed: client connection to 127.17.188.125:45009: connect: Connection refused (error 111)
I20260620 23:06:13.648586 31574 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.125:45009
I20260620 23:06:13.648618 31641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.125:45009 every 8 connection(s)
I20260620 23:06:13.648950 31574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
I20260620 23:06:13.649811 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 31574
I20260620 23:06:13.649813 31642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:13.649910 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-1/wal/instance
I20260620 23:06:13.650086 18161 external_mini_cluster.cc:1428] Setting key a162c32816bf134afb81c1dc8fbda2fa
I20260620 23:06:13.651315 31642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.188.124@KRBTEST.COM; defaulting to no policy
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.124@KRBTEST.COM" created.
I20260620 23:06:13.660322 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.651915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45543 (local address 127.0.0.1:55508)
0620 23:06:13.652075 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.652123 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:13.652613 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:13.652837 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:13.652840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:13.653065 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:13.653250 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.653254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.653961 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.653963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.654914 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.654917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.655354 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.655355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.655470 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.655974 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:13.655982 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:13.657469 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:13.658863 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.658870 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.658873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.659196 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.659410 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.659411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.659412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.659538 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.659832 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:13.659839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:13.659982 (+ 143us) client_negotiation.cc:769] Sending connection context
0620 23:06:13.660069 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 23:06:13.660170 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260620 23:06:13.660380 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.651955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.17.188.126:45543)
0620 23:06:13.652012 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.652015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.652248 (+ 233us) server_negotiation.cc:407] Connection header received
0620 23:06:13.652703 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.652705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.652716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.652790 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.653372 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.653857 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.655038 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.655256 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.655343 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.657549 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.657550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.657550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.657556 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.658793 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.659280 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.659281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.659282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.659339 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.659600 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.659601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.659601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.659737 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.659822 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.660111 (+ 289us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.660249 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:06:13.660955 31642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
I20260620 23:06:13.668339 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.661341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:45009 (local address 127.0.0.1:52000)
0620 23:06:13.661388 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.661399 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:13.661446 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:13.661716 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:13.661716 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:13.661814 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:13.661861 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.661862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.663116 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.663117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.663541 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.663542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.664184 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.664185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.664229 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.664635 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:13.664639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:13.665788 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:13.667484 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.667485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.667485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.667635 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.667805 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.667805 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.667805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.667888 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.668139 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:13.668140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:13.668172 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 23:06:13.668220 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 23:06:13.668286 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 23:06:13.668526 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.661408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52000 (local address 127.17.188.125:45009)
0620 23:06:13.661558 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.661563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.661585 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:06:13.661606 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.661608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.661636 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.661707 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.661928 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.663037 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.663616 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.664112 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.664189 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.665869 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.665876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.665876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.665883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.667414 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.667702 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.667704 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.667704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.667751 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.667957 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.667958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.667959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.668082 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.668141 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.668241 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.668395 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260620 23:06:13.669095 31642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:06:13.669425 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.669282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47364)
0620 23:06:13.669361 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.669407 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260620 23:06:13.669591 31592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
I20260620 23:06:13.669977 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.669901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47370)
0620 23:06:13.669942 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.669963 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:06:13.670087 31642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:37183: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab.
Entry for principal kudu/127.17.188.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab.
I20260620 23:06:13.673918 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.124:37183
--webserver_interface=127.17.188.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab
--principal=kudu/127.17.188.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:45543,127.17.188.125:45009,127.17.188.124:37183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:13.719314 31642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } attempt: 1
I20260620 23:06:13.719802 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.719649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47372)
0620 23:06:13.719712 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.719767 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 23:06:13.720292 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.720151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47374)
0620 23:06:13.720198 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.720231 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:06:13.720459 31642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:37183: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
I20260620 23:06:13.738940 31567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } attempt: 3
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
I20260620 23:06:13.745904 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.739293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52010 (local address 127.17.188.125:45009)
0620 23:06:13.739358 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.739360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.739369 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:06:13.739395 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.739396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.739407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.739475 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.739787 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.740274 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.740932 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.741135 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.741208 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.743344 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.743345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.743346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.743358 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.744768 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.745136 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.745137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.745137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.745196 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.745396 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.745397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.745398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.745506 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.745580 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.745750 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.745821 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:06:13.745918 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.739231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:45009 (local address 127.0.0.1:52010)
0620 23:06:13.739269 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.739280 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:13.739331 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:13.739482 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:13.739483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:13.739642 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:13.739701 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.739704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.740381 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.740382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.740849 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.740851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.741207 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.741208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.741287 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.741760 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:13.741765 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:13.743247 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:13.744891 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.744893 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.744894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.745063 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.745264 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.745265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.745266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.745346 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.745584 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:13.745585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:13.745629 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 23:06:13.745714 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 23:06:13.745814 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260620 23:06:13.746510 31567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:06:13.746824 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.746726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47386)
0620 23:06:13.746777 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.746802 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 23:06:13.747133 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.747050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47402)
0620 23:06:13.747091 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.747112 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 23:06:13.747221 31567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:37183: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
W20260620 23:06:13.754086 31649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:13.754282 31649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:13.754328 31649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:13.755448 31649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:13.755511 31649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:13.755538 31649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:13.755578 31649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:13.755609 31649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:13.757084 31649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:45543,127.17.188.125:45009,127.17.188.124:37183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab
--principal=kudu/127.17.188.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.124:37183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.188.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:13.757375 31649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:13.757579 31649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:13.762024 31654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:13.762176 31655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:13.762306 31649 server_base.cc:1034] running on GCE node
W20260620 23:06:13.762344 31657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:13.762635 31649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:13.762931 31649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:13.764087 31649 hybrid_clock.cc:648] HybridClock initialized: now 1781996773764083 us; error 32 us; skew 500 ppm
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:13.766398 31649 init.cc:377] Logged in from keytab as kudu/127.17.188.124@KRBTEST.COM (short username kudu)
I20260620 23:06:13.766826 31649 webserver.cc:460] Webserver started at http://127.17.188.124:43365/ using document root <none> and password file <none>
I20260620 23:06:13.767045 31649 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:13.767088 31649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:13.767239 31649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:13.768288 31649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/data/instance:
uuid: "00927aef0dc84ae48b0ef88e4605497e"
format_stamp: "Formatted at 2026-06-20 23:06:13 on dist-test-slave-zpss"
server_key: "3199572d5ea0f210613809e392527c9b"
server_key_iv: "891ca8283219666250fbe6dc69a4d663"
server_key_version: "encryptionkey@0"
I20260620 23:06:13.768635 31649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/wal/instance:
uuid: "00927aef0dc84ae48b0ef88e4605497e"
format_stamp: "Formatted at 2026-06-20 23:06:13 on dist-test-slave-zpss"
server_key: "3199572d5ea0f210613809e392527c9b"
server_key_iv: "891ca8283219666250fbe6dc69a4d663"
server_key_version: "encryptionkey@0"
I20260620 23:06:13.770017 31649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:06:13.770664 31664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:13.770829 31649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:13.770901 31649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/wal
uuid: "00927aef0dc84ae48b0ef88e4605497e"
format_stamp: "Formatted at 2026-06-20 23:06:13 on dist-test-slave-zpss"
server_key: "3199572d5ea0f210613809e392527c9b"
server_key_iv: "891ca8283219666250fbe6dc69a4d663"
server_key_version: "encryptionkey@0"
I20260620 23:06:13.770988 31649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:13.778344 31567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } attempt: 1
I20260620 23:06:13.778661 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.778554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47406)
0620 23:06:13.778605 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.778637 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 23:06:13.778956 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.778868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47412)
0620 23:06:13.778909 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.778932 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 23:06:13.779045 31567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:37183: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
I20260620 23:06:13.785086 31649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:13.787400 31649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:13.787557 31649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:13.788596 31642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } attempt: 2
I20260620 23:06:13.788944 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.788825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47422)
0620 23:06:13.788888 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.788921 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 23:06:13.789371 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.789205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47430)
0620 23:06:13.789276 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.789331 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 23:06:13.789482 31642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:37183: Network error: Client connection negotiation failed: client connection to 127.17.188.124:37183: connect: Connection refused (error 111)
I20260620 23:06:13.791661 31649 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.124:37183
I20260620 23:06:13.791687 31716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.124:37183 every 8 connection(s)
I20260620 23:06:13.792037 31649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
I20260620 23:06:13.792853 31717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:13.794440 31717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:06:13.798332 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 31649
I20260620 23:06:13.798466 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996599092328-18161-0/minicluster-data/master-2/wal/instance
I20260620 23:06:13.798717 18161 external_mini_cluster.cc:1428] Setting key 1bb37d07748ad83a4b1223c9b87856b1
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:13.803136 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.795093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.17.188.126:45543)
0620 23:06:13.795156 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.795158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.795365 (+ 207us) server_negotiation.cc:407] Connection header received
0620 23:06:13.795793 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.795793 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.795801 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.795856 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.796397 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.796845 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.797796 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.797965 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.798015 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.800201 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.800203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.800203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.800210 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.801697 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.802157 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.802158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.802159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.802204 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.802445 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.802445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.802446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.802568 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.802636 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.802921 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.803016 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:06:13.803295 31719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.795084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45543 (local address 127.0.0.1:55524)
0620 23:06:13.795234 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.795274 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:13.795717 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:13.795922 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:13.795925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:13.796148 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:13.796322 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.796326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.796919 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.796920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.797728 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.797731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.798033 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.798034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.798125 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.798660 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:13.798674 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:13.800137 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:13.801773 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.801780 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.801782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.802084 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.802268 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.802269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.802269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.802386 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.802634 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:13.802638 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:13.802770 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 23:06:13.802899 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 23:06:13.803065 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
I20260620 23:06:13.803937 31717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
I20260620 23:06:13.808668 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.799117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47446 (local address 127.17.188.124:37183)
0620 23:06:13.799289 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.799294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.799317 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:06:13.799379 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.799382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.799402 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.799473 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.800133 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.801302 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.802291 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.802683 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.802742 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.805430 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.805437 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.805438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.805444 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.807160 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.807545 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.807548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.807548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.807626 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.807856 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.807857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.807857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.807989 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.808142 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.808462 (+ 320us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.808544 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260620 23:06:13.810316 31719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.804180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:45009 (local address 127.0.0.1:52020)
0620 23:06:13.804306 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.804317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:13.804372 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:13.804504 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:13.804505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:13.804607 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:13.804647 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.804648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.805415 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.805416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.805902 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.805904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.806372 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.806373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.806441 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.806754 (+ 313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:13.806757 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:13.807793 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:13.809293 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.809294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.809294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.809505 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.809710 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.809710 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.809711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.809767 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.810052 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:13.810053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:13.810082 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 23:06:13.810127 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 23:06:13.810233 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260620 23:06:13.810354 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.804253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.17.188.125:45009)
0620 23:06:13.804315 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.804317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.804360 (+ 43us) server_negotiation.cc:407] Connection header received
0620 23:06:13.804431 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.804432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.804443 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.804505 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.804716 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.805307 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.805963 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.806281 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.806372 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.807897 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.807899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.807899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.807905 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.809206 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.809573 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.809574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.809575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.809635 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.809833 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.809833 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.809834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.809983 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.810059 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.810167 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.810261 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:06:13.810704 31717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
I20260620 23:06:13.816416 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.809549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55538 (local address 127.17.188.126:45543)
0620 23:06:13.809589 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.809591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.809597 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:06:13.809614 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.809614 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.809621 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.809670 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.810232 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.810846 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.811460 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.811735 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.811805 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.813821 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.813823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.813823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.813829 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.815265 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.815682 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.815683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.815683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.815730 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.815927 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.815928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.815928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.816013 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.816125 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.816243 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.816318 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:06:13.817405 31719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.810917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47462)
0620 23:06:13.810968 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.810977 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:13.811014 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:13.811129 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:13.811130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:13.811250 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:13.811270 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.811271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.812051 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.812052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.812505 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.812506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.812870 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.812871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.812928 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.813291 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:13.813296 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:13.814641 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:13.816229 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.816230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.816230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.816443 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.816721 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.816722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.816722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.816799 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.817084 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:13.817085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:13.817120 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 23:06:13.817195 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 23:06:13.817281 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 23:06:13.817467 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.810968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47462 (local address 127.17.188.124:37183)
0620 23:06:13.811000 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.811002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.811009 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:06:13.811067 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.811067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.811076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.811127 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.811339 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.811982 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.812594 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.812789 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.812866 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.814712 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.814713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.814714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.814719 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.816156 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.816520 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.816521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.816521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.816571 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.816865 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.816866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.816866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.816968 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.817045 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.817216 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.817351 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":20}
I20260620 23:06:13.819005 31717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e: Bootstrap starting.
I20260620 23:06:13.819998 31717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:13.820279 31717 log.cc:824] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:13.821062 31717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e: No bootstrap required, opened a new log
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
I20260620 23:06:13.822899 31717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } }
I20260620 23:06:13.823064 31717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:13.823117 31717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00927aef0dc84ae48b0ef88e4605497e, State: Initialized, Role: FOLLOWER
I20260620 23:06:13.823238 31717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } }
I20260620 23:06:13.823536 31723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:13.823582 31717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:13.823635 31723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:06:13.824121 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.817045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52034 (local address 127.17.188.125:45009)
0620 23:06:13.817084 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.817086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.817094 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:06:13.817114 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.817115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.817123 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.817164 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.817786 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.818478 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.819112 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.819387 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.819469 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.821696 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.821697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.821697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.821702 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.822936 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.823307 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.823308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.823308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.823383 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.823586 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.823588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.823588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.823713 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.823891 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.823928 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.824004 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:06:13.825448 31734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:06:13.825512 31734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:06:13.826115 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:06:13.825680 31649 master_runner.cc:186] Error getting master registration for 127.17.188.126:45543: OK, instance_id { permanent_uuid: "490b32cf099f44e584c24a02209397a5" instance_seqno: 1781996773509718 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:06:13.828275 31649 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:06:13.834986 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.827629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55544 (local address 127.17.188.126:45543)
0620 23:06:13.827663 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.827665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.827679 (+ 14us) server_negotiation.cc:407] Connection header received
0620 23:06:13.827741 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.827742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.827752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.827809 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.828678 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.829205 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.830230 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.830516 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.830552 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.831922 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.831923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.831923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.831930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.833728 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.834124 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.834125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.834126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.834179 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.834422 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.834423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.834423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.834542 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.834627 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.834816 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.834890 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:06:13.835347 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.827683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47464 (local address 127.17.188.124:37183)
0620 23:06:13.827715 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.827717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.828103 (+ 386us) server_negotiation.cc:407] Connection header received
0620 23:06:13.828188 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.828189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.828197 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.828225 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.828876 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.829570 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.830547 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.830803 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.830840 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.832349 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.832350 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.832351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.832358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.834037 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.834469 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.834470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.834470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.834520 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.834731 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.834732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.834733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.834892 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.834971 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.835183 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.835257 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:06:13.836578 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.827872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52046 (local address 127.17.188.125:45009)
0620 23:06:13.827900 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.827902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.829442 (+ 1540us) server_negotiation.cc:407] Connection header received
0620 23:06:13.829527 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.829528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.829536 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.829562 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.830230 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.830815 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.831927 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.832209 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.832266 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.833485 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.833487 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.833487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.833494 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.835125 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.835953 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.835955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.835955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.836009 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.836203 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.836203 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.836203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.836283 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.836338 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.836480 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.836529 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:06:13.854182 31567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } attempt: 2
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
I20260620 23:06:13.860729 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.854467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47472)
0620 23:06:13.854534 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.854542 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:13.854592 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:13.854725 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:13.854726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:13.854927 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:13.854972 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.854974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.855728 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.855729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.856195 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.856197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.856613 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.856614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.856670 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.857034 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:13.857039 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:13.858181 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:13.859787 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.859788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.859788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.859935 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.860126 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.860127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.860127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.860186 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.860440 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:13.860441 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:13.860481 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 23:06:13.860544 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 23:06:13.860630 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 23:06:13.861061 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.854555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47472 (local address 127.17.188.124:37183)
0620 23:06:13.854613 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.854615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.854623 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:06:13.854642 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.854643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.854651 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.854722 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.855168 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.855638 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.856275 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.856506 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.856602 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.858258 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.858259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.858260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.858266 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.859704 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.860000 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.860001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.860001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.860055 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.860247 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.860248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.860248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.860360 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.860423 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.860739 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.861012 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:06:13.862603 31567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5: Bootstrap starting.
I20260620 23:06:13.863279 31567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:13.863590 31567 log.cc:824] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:13.864349 31567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5: No bootstrap required, opened a new log
I20260620 23:06:13.866348 31567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } }
I20260620 23:06:13.866478 31567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:13.866510 31567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490b32cf099f44e584c24a02209397a5, State: Initialized, Role: FOLLOWER
I20260620 23:06:13.866645 31567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } }
I20260620 23:06:13.866894 31744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:13.866938 31567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:13.866997 31744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:06:13.868681 31755 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:06:13.868775 31755 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:06:13.869299 31499 master_runner.cc:186] Error getting master registration for 127.17.188.125:45009: OK, instance_id { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" instance_seqno: 1781996773647276 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:06:13.871390 31499 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:06:13.880636 31642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } attempt: 3
Jun 20 23:06:13 dist-test-slave-zpss krb5kdc[31486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
I20260620 23:06:13.888348 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.880857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:37183 (local address 127.0.0.1:47484)
0620 23:06:13.880897 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:13.880906 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:13.880937 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:13.881068 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:13.881069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:13.881246 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:13.881287 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.881289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.882377 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.882377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.883123 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:13.883125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.883600 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.883600 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:13.883658 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.884108 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:13.884113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:13.885590 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:13.887332 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.887333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.887334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.887540 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.887746 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:13.887746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:13.887747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:13.887809 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:13.888062 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:13.888063 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:13.888104 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 23:06:13.888172 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:06:13.888255 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260620 23:06:13.888360 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:13.880919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.17.188.124:37183)
0620 23:06:13.880958 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:13.880960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:13.880969 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:06:13.880987 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:13.880988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:13.880995 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:13.881052 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:13.881390 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.882082 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.883209 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:13.883518 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:13.883589 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:13.885678 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:13.885680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:13.885680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:13.885687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:13.887244 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.887616 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.887617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.887618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.887674 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:13.887878 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:13.887879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:13.887879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:13.887995 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:13.888059 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:13.888209 (+ 150us) server_negotiation.cc:299] Negotiation successful
0620 23:06:13.888277 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:06:13.890017 31642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf: Bootstrap starting.
I20260620 23:06:13.890611 31642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:13.890894 31642 log.cc:824] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:13.891649 31642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf: No bootstrap required, opened a new log
I20260620 23:06:13.893509 31642 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } }
I20260620 23:06:13.893641 31642 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:13.893673 31642 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5a9bccea26e40eca7b9d488d6ea9edf, State: Initialized, Role: FOLLOWER
I20260620 23:06:13.893778 31642 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } }
I20260620 23:06:13.894070 31642 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:13.894042 31758 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:13.894207 31758 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:06:13.894901 31595 tablet.cc:2369] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:06:13.895773 31769 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:06:13.895887 31769 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:06:13.898566 31574 master_runner.cc:263] No leader master found from master a5a9bccea26e40eca7b9d488d6ea9edf
I20260620 23:06:13.898615 31574 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 23:06:14.007609 31520 tablet.cc:2369] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:06:14.038208 31670 tablet.cc:2369] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:06:14.088438 31758 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:14.088547 31758 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } }
I20260620 23:06:14.088858 31758 leader_election.cc:288] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 490b32cf099f44e584c24a02209397a5 (127.17.188.126:45543), 00927aef0dc84ae48b0ef88e4605497e (127.17.188.124:37183)
I20260620 23:06:14.089072 31691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00927aef0dc84ae48b0ef88e4605497e" is_pre_election: true
I20260620 23:06:14.089136 31541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490b32cf099f44e584c24a02209397a5" is_pre_election: true
I20260620 23:06:14.089247 31691 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5a9bccea26e40eca7b9d488d6ea9edf in term 0.
I20260620 23:06:14.089315 31541 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5a9bccea26e40eca7b9d488d6ea9edf in term 0.
I20260620 23:06:14.089461 31592 leader_election.cc:302] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00927aef0dc84ae48b0ef88e4605497e, a5a9bccea26e40eca7b9d488d6ea9edf; no voters:
I20260620 23:06:14.089593 31758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:06:14.089694 31758 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:06:14.089731 31758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:14.090472 31758 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } }
I20260620 23:06:14.090626 31758 leader_election.cc:288] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [CANDIDATE]: Term 1 election: Requested vote from peers 490b32cf099f44e584c24a02209397a5 (127.17.188.126:45543), 00927aef0dc84ae48b0ef88e4605497e (127.17.188.124:37183)
I20260620 23:06:14.090780 31691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00927aef0dc84ae48b0ef88e4605497e"
I20260620 23:06:14.090762 31541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490b32cf099f44e584c24a02209397a5"
I20260620 23:06:14.090862 31691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:14.090845 31541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:14.091529 31691 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5a9bccea26e40eca7b9d488d6ea9edf in term 1.
I20260620 23:06:14.091529 31541 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5a9bccea26e40eca7b9d488d6ea9edf in term 1.
I20260620 23:06:14.091707 31592 leader_election.cc:302] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00927aef0dc84ae48b0ef88e4605497e, a5a9bccea26e40eca7b9d488d6ea9edf; no voters:
I20260620 23:06:14.091809 31758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:14.092006 31758 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [term 1 LEADER]: Becoming Leader. State: Replica: a5a9bccea26e40eca7b9d488d6ea9edf, State: Running, Role: LEADER
I20260620 23:06:14.092118 31758 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } }
I20260620 23:06:14.092514 31771 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5a9bccea26e40eca7b9d488d6ea9edf. Latest consensus state: current_term: 1 leader_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:14.092623 31771 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:14.092808 31773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:06:14.092978 31773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:06:14.094288 31691 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [term 1 FOLLOWER]: Refusing update from remote peer a5a9bccea26e40eca7b9d488d6ea9edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:14.094303 31541 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [term 1 FOLLOWER]: Refusing update from remote peer a5a9bccea26e40eca7b9d488d6ea9edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:14.094671 31758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:14.094799 31771 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:14.095978 31723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5a9bccea26e40eca7b9d488d6ea9edf. Latest consensus state: current_term: 1 leader_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:14.096058 31723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:06:14.096939 31771 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:14.096930 31744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5a9bccea26e40eca7b9d488d6ea9edf. Latest consensus state: current_term: 1 leader_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:14.097005 31771 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:14.097005 31744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:06:14.097118 31723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:14.097208 31723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:06:14.097306 31774 mvcc.cc:205] Tried to move back new op lower bound from 7299058786688196608 to 7299058786682003456. Current Snapshot: MvccSnapshot[applied={T|T < 7299058786688196608}]
I20260620 23:06:14.097374 31775 mvcc.cc:205] Tried to move back new op lower bound from 7299058786688196608 to 7299058786682003456. Current Snapshot: MvccSnapshot[applied={T|T < 7299058786688196608}]
I20260620 23:06:14.097692 31758 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:14.097781 31758 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:14.097956 31773 catalog_manager.cc:1347] Generated new cluster ID: 01917f4503a14da89e4e61571ba42837
I20260620 23:06:14.098012 31773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:06:14.098076 31744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490b32cf099f44e584c24a02209397a5" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45543 } } peers { permanent_uuid: "a5a9bccea26e40eca7b9d488d6ea9edf" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 45009 } } peers { permanent_uuid: "00927aef0dc84ae48b0ef88e4605497e" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 37183 } } }
I20260620 23:06:14.098153 31744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:06:14.116578 31773 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:06:14.117201 31773 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:06:14.124117 31773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5a9bccea26e40eca7b9d488d6ea9edf: Generated new TSK 0
I20260620 23:06:14.124302 31773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:06:14.825947 31734 catalog_manager.cc:1259] Loaded cluster ID: 01917f4503a14da89e4e61571ba42837
I20260620 23:06:14.826248 31734 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e: loading cluster ID for follower catalog manager: success
I20260620 23:06:14.827716 31734 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e: acquiring CA information for follower catalog manager: success
I20260620 23:06:14.828059 31734 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 00927aef0dc84ae48b0ef88e4605497e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:06:14.869202 31755 catalog_manager.cc:1259] Loaded cluster ID: 01917f4503a14da89e4e61571ba42837
I20260620 23:06:14.869336 31755 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5: loading cluster ID for follower catalog manager: success
I20260620 23:06:14.870115 31755 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5: acquiring CA information for follower catalog manager: success
I20260620 23:06:14.870421 31755 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 490b32cf099f44e584c24a02209397a5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:06:15.179843 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 31499
I20260620 23:06:15.184897 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 31574
I20260620 23:06:15.189628 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 31649
2026-06-20T23:06:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2927 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:06:15 dist-test-slave-zpss krb5kdc[31789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:06:15 dist-test-slave-zpss krb5kdc[31789](info): set up 2 sockets
Jun 20 23:06:15 dist-test-slave-zpss krb5kdc[31789](info): commencing operation
krb5kdc: starting...
W20260620 23:06:16.213538 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:06:16 dist-test-slave-zpss krb5kdc[31789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:06:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:06:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:06:16.320782 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34899
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:34899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:16.396162 31802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:16.396382 31802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:16.396422 31802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:16.397550 31802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:16.397619 31802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:16.397643 31802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:16.397671 31802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:16.397701 31802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:16.399133 31802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:34899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:16.399397 31802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:16.399611 31802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:16.404165 31808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:16.404228 31802 server_base.cc:1034] running on GCE node
W20260620 23:06:16.404277 31810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:16.404390 31807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:16.404670 31802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:16.404999 31802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:16.406157 31802 hybrid_clock.cc:648] HybridClock initialized: now 1781996776406151 us; error 40 us; skew 500 ppm
Jun 20 23:06:16 dist-test-slave-zpss krb5kdc[31789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996776, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:16.408517 31802 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:06:16.408931 31802 webserver.cc:460] Webserver started at http://127.17.188.126:40247/ using document root <none> and password file <none>
I20260620 23:06:16.409157 31802 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:16.409224 31802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:16.409408 31802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:16.410430 31802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "1aeb4147c839406e875a9f1c41606a32"
format_stamp: "Formatted at 2026-06-20 23:06:16 on dist-test-slave-zpss"
server_key: "aa088f60c22b5be2a93a2f8337ef42a6"
server_key_iv: "eef7f5181565b23e3734b8020dd2b204"
server_key_version: "encryptionkey@0"
I20260620 23:06:16.410825 31802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "1aeb4147c839406e875a9f1c41606a32"
format_stamp: "Formatted at 2026-06-20 23:06:16 on dist-test-slave-zpss"
server_key: "aa088f60c22b5be2a93a2f8337ef42a6"
server_key_iv: "eef7f5181565b23e3734b8020dd2b204"
server_key_version: "encryptionkey@0"
I20260620 23:06:16.412067 31802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:06:16.412732 31817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:16.412909 31802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:16.413007 31802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "1aeb4147c839406e875a9f1c41606a32"
format_stamp: "Formatted at 2026-06-20 23:06:16 on dist-test-slave-zpss"
server_key: "aa088f60c22b5be2a93a2f8337ef42a6"
server_key_iv: "eef7f5181565b23e3734b8020dd2b204"
server_key_version: "encryptionkey@0"
I20260620 23:06:16.413079 31802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:16.430647 31802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:16.432332 31802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:16.432498 31802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:16.436367 31802 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:34899
I20260620 23:06:16.436394 31869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:34899 every 8 connection(s)
I20260620 23:06:16.436721 31802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:06:16.437614 31870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:16.439656 31870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32: Bootstrap starting.
I20260620 23:06:16.440187 31870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:16.440462 31870 log.cc:824] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:16.441112 31870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32: No bootstrap required, opened a new log
I20260620 23:06:16.442665 31870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb4147c839406e875a9f1c41606a32" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34899 } }
I20260620 23:06:16.442792 31870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:16.442813 31870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aeb4147c839406e875a9f1c41606a32, State: Initialized, Role: FOLLOWER
I20260620 23:06:16.442981 31870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb4147c839406e875a9f1c41606a32" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34899 } }
I20260620 23:06:16.443058 31870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:06:16.443120 31870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:06:16.443172 31870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:16.443676 31870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb4147c839406e875a9f1c41606a32" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34899 } }
I20260620 23:06:16.443780 31870 leader_election.cc:302] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aeb4147c839406e875a9f1c41606a32; no voters:
I20260620 23:06:16.443964 31870 leader_election.cc:288] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:06:16.443991 31873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:16.444154 31873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [term 1 LEADER]: Becoming Leader. State: Replica: 1aeb4147c839406e875a9f1c41606a32, State: Running, Role: LEADER
I20260620 23:06:16.444206 31870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:16.444285 31873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb4147c839406e875a9f1c41606a32" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34899 } }
I20260620 23:06:16.444662 31875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aeb4147c839406e875a9f1c41606a32. Latest consensus state: current_term: 1 leader_uuid: "1aeb4147c839406e875a9f1c41606a32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb4147c839406e875a9f1c41606a32" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34899 } } }
I20260620 23:06:16.444787 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 31802
I20260620 23:06:16.444787 31874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aeb4147c839406e875a9f1c41606a32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb4147c839406e875a9f1c41606a32" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34899 } } }
I20260620 23:06:16.444849 31874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:16.444818 31875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32 [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:16.444900 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:06:16.445061 31882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:06:16.445097 18161 external_mini_cluster.cc:1428] Setting key 8022a54ae80171c8831005a91dc5688c
I20260620 23:06:16.445586 31882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:06:16.447165 31882 catalog_manager.cc:1347] Generated new cluster ID: bb464fb61aba4d3e9d56994013f372b2
I20260620 23:06:16.447214 31882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:06:16.452057 31882 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:06:16.452521 31882 catalog_manager.cc:1504] Loading token signing keys...
Jun 20 23:06:16 dist-test-slave-zpss krb5kdc[31789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:16.455483 31882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aeb4147c839406e875a9f1c41606a32: Generated new TSK 0
I20260620 23:06:16.455677 31882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:06:16.457444 31889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:16.445519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.17.188.126:34899)
0620 23:06:16.445907 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:16.445917 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:16.446134 (+ 217us) server_negotiation.cc:407] Connection header received
0620 23:06:16.446288 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:16.446296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:16.446350 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:16.446482 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:16.447305 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:16.448603 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:16.449670 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:16.450296 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:16.450392 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:16.453179 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:16.453183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:16.453183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:16.453194 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:16.455833 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:16.456152 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:16.456155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:16.456156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:16.456275 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:16.456467 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:16.456468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:16.456468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:16.456745 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:16.456873 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:16.456973 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 23:06:16.457218 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":76,"threads_started":1}
I20260620 23:06:16.458252 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:06:16.460805 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 31802
2026-06-20T23:06:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1270 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:06:16 dist-test-slave-zpss krb5kdc[31897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:06:16 dist-test-slave-zpss krb5kdc[31897](info): set up 2 sockets
Jun 20 23:06:16 dist-test-slave-zpss krb5kdc[31897](info): commencing operation
krb5kdc: starting...
W20260620 23:06:17.457669 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:06:17 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:06:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:06:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:06:17.568411 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45551
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:45551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:17.644820 31910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:17.645059 31910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:17.645100 31910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:17.646229 31910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:17.646296 31910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:17.646318 31910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:17.646338 31910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:17.646368 31910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:17.647765 31910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:45551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:17.648030 31910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:17.648236 31910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:17.652596 31916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:17.652596 31918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:17.652751 31910 server_base.cc:1034] running on GCE node
W20260620 23:06:17.652626 31915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:17.653129 31910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:17.653458 31910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:17.654600 31910 hybrid_clock.cc:648] HybridClock initialized: now 1781996777654588 us; error 27 us; skew 500 ppm
Jun 20 23:06:17 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996777, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:17.656913 31910 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:06:17.657352 31910 webserver.cc:460] Webserver started at http://127.17.188.126:42067/ using document root <none> and password file <none>
I20260620 23:06:17.657541 31910 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:17.657579 31910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:17.657661 31910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:17.658496 31910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "523c075900a947dc9016a4330d826d6e"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "d17e8788e00d2e477a0a5f8552a38124"
server_key_iv: "4ac0978911ef0673053c9fa561038e88"
server_key_version: "encryptionkey@0"
I20260620 23:06:17.658756 31910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "523c075900a947dc9016a4330d826d6e"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "d17e8788e00d2e477a0a5f8552a38124"
server_key_iv: "4ac0978911ef0673053c9fa561038e88"
server_key_version: "encryptionkey@0"
I20260620 23:06:17.659821 31910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:06:17.660393 31925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:17.660552 31910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:06:17.660636 31910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "523c075900a947dc9016a4330d826d6e"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "d17e8788e00d2e477a0a5f8552a38124"
server_key_iv: "4ac0978911ef0673053c9fa561038e88"
server_key_version: "encryptionkey@0"
I20260620 23:06:17.660703 31910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:17.673687 31910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:17.675278 31910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:17.675422 31910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:17.679095 31910 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:45551
I20260620 23:06:17.679122 31977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:45551 every 8 connection(s)
I20260620 23:06:17.679404 31910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:06:17.680303 31978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:17.682000 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 31910
I20260620 23:06:17.682108 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:06:17.682296 18161 external_mini_cluster.cc:1428] Setting key fb54ada2ca27046d502075af7889ab0e
I20260620 23:06:17.683136 31978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Bootstrap starting.
I20260620 23:06:17.683761 31978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:17.684094 31978 log.cc:824] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:17.684834 31978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: No bootstrap required, opened a new log
I20260620 23:06:17.686991 31978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:17.687176 31978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:17.687220 31978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 523c075900a947dc9016a4330d826d6e, State: Initialized, Role: FOLLOWER
I20260620 23:06:17.687330 31978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:17.687420 31978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:06:17.687451 31978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:06:17.687490 31978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:17.688140 31978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:17.688256 31978 leader_election.cc:302] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 523c075900a947dc9016a4330d826d6e; no voters:
I20260620 23:06:17.688422 31978 leader_election.cc:288] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:06:17.688548 31983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:17.688651 31978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:17.688819 31983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 1 LEADER]: Becoming Leader. State: Replica: 523c075900a947dc9016a4330d826d6e, State: Running, Role: LEADER
I20260620 23:06:17.688998 31983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
Jun 20 23:06:17 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:17.690383 31985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 523c075900a947dc9016a4330d826d6e. Latest consensus state: current_term: 1 leader_uuid: "523c075900a947dc9016a4330d826d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } } }
I20260620 23:06:17.690485 31985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:17.690730 31997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:06:17.690747 31984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "523c075900a947dc9016a4330d826d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } } }
I20260620 23:06:17.691320 31984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:17.691341 31997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:06:17.692840 31997 catalog_manager.cc:1347] Generated new cluster ID: 31c4c220998a4f40a7b859d743c4eb3e
I20260620 23:06:17.692910 31997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:06:17.693871 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:17.682772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58560 (local address 127.17.188.126:45551)
0620 23:06:17.682913 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:17.682921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:17.682958 (+ 37us) server_negotiation.cc:407] Connection header received
0620 23:06:17.683405 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:17.683411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:17.683461 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:17.683564 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:17.684362 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:17.685449 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:17.686489 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:17.686845 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:17.686912 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:17.690155 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:17.690159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:17.690159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:17.690170 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:17.692182 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:17.692557 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:17.692560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:17.692560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:17.692662 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:17.692986 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:17.692987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:17.692988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:17.693277 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:17.693466 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:17.693559 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 23:06:17.693689 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260620 23:06:17.726280 31997 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:06:17.726796 31997 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:06:17.734486 31997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Generated new TSK 0
I20260620 23:06:17.734668 31997 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:06:17.767802 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45551
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:17.855564 32004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:17.855755 32004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:17.855777 32004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:17.856815 32004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:17.856848 32004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:17.856873 32004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:06:17.858296 32004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:17.858464 32004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:17.858619 32004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:17.863221 32009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:17.863268 32012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:17.863302 32010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:17.863390 32004 server_base.cc:1034] running on GCE node
I20260620 23:06:17.863550 32004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:17.863787 32004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:17.864941 32004 hybrid_clock.cc:648] HybridClock initialized: now 1781996777864926 us; error 27 us; skew 500 ppm
Jun 20 23:06:17 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996777, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:17.867432 32004 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:06:17.867789 32004 webserver.cc:460] Webserver started at http://127.17.188.65:41487/ using document root <none> and password file <none>
I20260620 23:06:17.867950 32004 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:17.867987 32004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:17.868079 32004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:17.868999 32004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "378924fa0b92429fa1e0576446721836"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "c0b595daed0ff6d31b401d46b6a42dd6"
server_key_iv: "214545342f66ab82a62eacda668dfb61"
server_key_version: "encryptionkey@0"
I20260620 23:06:17.869305 32004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "378924fa0b92429fa1e0576446721836"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "c0b595daed0ff6d31b401d46b6a42dd6"
server_key_iv: "214545342f66ab82a62eacda668dfb61"
server_key_version: "encryptionkey@0"
I20260620 23:06:17.870571 32004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:17.871186 32019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:17.871341 32004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:17.871421 32004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "378924fa0b92429fa1e0576446721836"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "c0b595daed0ff6d31b401d46b6a42dd6"
server_key_iv: "214545342f66ab82a62eacda668dfb61"
server_key_version: "encryptionkey@0"
I20260620 23:06:17.871474 32004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:17.881104 32004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:17.883163 32004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:17.883261 32004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:17.883405 32004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:17.883643 32004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:17.883669 32004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:17.883687 32004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:17.883698 32004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:17.889086 32004 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:36745
I20260620 23:06:17.889155 32132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:36745 every 8 connection(s)
I20260620 23:06:17.889477 32004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:06:17.892637 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 32004
I20260620 23:06:17.892742 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:06:17.892894 18161 external_mini_cluster.cc:1428] Setting key ea9fbff0c725dcf9316a376c9c8e07fc
Jun 20 23:06:17 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996777, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:17.898979 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:17.890191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:43671 (local address 127.17.188.126:45551)
0620 23:06:17.890266 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:17.890268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:17.890404 (+ 136us) server_negotiation.cc:407] Connection header received
0620 23:06:17.890939 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:17.890940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:17.890950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:17.891012 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:17.891829 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:17.892293 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:17.893624 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:17.893953 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:17.894034 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:17.896247 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:17.896248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:17.896249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:17.896256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:17.897657 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:17.898060 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:17.898062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:17.898062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:17.898118 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:17.898353 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:17.898354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:17.898354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:17.898456 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:17.898545 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:17.898799 (+ 254us) server_negotiation.cc:299] Negotiation successful
0620 23:06:17.898881 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 23:06:17.899156 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:17.890114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:43671)
0620 23:06:17.890284 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:17.890321 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:17.890810 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:17.891069 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:17.891072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:17.891581 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:17.891750 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:17.891754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:17.892375 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:17.892375 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:17.893543 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:17.893547 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:17.894043 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:17.894043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:17.894144 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:17.894656 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:17.894664 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:17.896151 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:17.897767 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:17.897773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:17.897775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:17.897976 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:17.898181 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:17.898182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:17.898182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:17.898286 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:17.898537 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:17.898544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:17.898674 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 23:06:17.898778 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 23:06:17.898920 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260620 23:06:17.899654 32133 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45551
I20260620 23:06:17.899749 32133 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:17.899910 32133 heartbeater.cc:511] Master 127.17.188.126:45551 requested a full tablet report, sending...
I20260620 23:06:17.900421 31942 ts_manager.cc:194] Registered new tserver with Master: 378924fa0b92429fa1e0576446721836 (127.17.188.65:36745)
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
I20260620 23:06:17.901068 31942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:43671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:06:17.916566 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45551
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:17.993655 32138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:17.993829 32138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:17.993849 32138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:17.994947 32138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:17.994979 32138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:17.995004 32138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:06:17.996307 32138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:17.996469 32138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:17.996624 32138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:18.001084 32143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:18.001106 32146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:18.001107 32144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:18.001308 32138 server_base.cc:1034] running on GCE node
I20260620 23:06:18.001657 32138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:18.001895 32138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:18.003060 32138 hybrid_clock.cc:648] HybridClock initialized: now 1781996778003046 us; error 51 us; skew 500 ppm
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:18.005581 32138 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:06:18.006049 32138 webserver.cc:460] Webserver started at http://127.17.188.66:37211/ using document root <none> and password file <none>
I20260620 23:06:18.006269 32138 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:18.006335 32138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:18.006479 32138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:18.007444 32138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "fb7ab3cfb5564290ada93f93290bc40d"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "85cc28e43ca350eefe31cba61b771711"
server_key_iv: "e628661e2714710c30acf3cf565059f6"
server_key_version: "encryptionkey@0"
I20260620 23:06:18.007804 32138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "fb7ab3cfb5564290ada93f93290bc40d"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "85cc28e43ca350eefe31cba61b771711"
server_key_iv: "e628661e2714710c30acf3cf565059f6"
server_key_version: "encryptionkey@0"
I20260620 23:06:18.009092 32138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:06:18.009929 32153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.010087 32138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.010182 32138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "fb7ab3cfb5564290ada93f93290bc40d"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "85cc28e43ca350eefe31cba61b771711"
server_key_iv: "e628661e2714710c30acf3cf565059f6"
server_key_version: "encryptionkey@0"
I20260620 23:06:18.010267 32138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:18.036970 32138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:18.039264 32138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:18.039420 32138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:18.039650 32138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:18.039932 32138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:18.039983 32138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.040022 32138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:18.040050 32138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.045583 32138 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:33871
I20260620 23:06:18.045639 32266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:33871 every 8 connection(s)
I20260620 23:06:18.045971 32138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:06:18.051205 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 32138
I20260620 23:06:18.051314 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:06:18.051481 18161 external_mini_cluster.cc:1428] Setting key afe602ce16897ac4d41be18c315d3d3b
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:18.054940 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.046733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:44103 (local address 127.17.188.126:45551)
0620 23:06:18.046793 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.046795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.046959 (+ 164us) server_negotiation.cc:407] Connection header received
0620 23:06:18.047400 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.047401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.047410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.047464 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:18.048313 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.048770 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.049745 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.049911 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.049968 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.052187 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:18.052189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:18.052189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:18.052195 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:18.053566 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.053983 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.053985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.053985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.054043 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.054309 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.054309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.054310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.054420 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:18.054499 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.054762 (+ 263us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.054852 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:06:18.055119 32269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.046676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:44103)
0620 23:06:18.046823 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.046863 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.047320 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.047524 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.047527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.048063 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:18.048238 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.048242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.048845 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.048845 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.049653 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.049654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.049977 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.049978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.050071 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.050595 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:18.050603 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:18.052103 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:18.053673 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.053680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.053682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.053903 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.054112 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.054113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.054113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.054239 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.054491 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:18.054497 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:18.054633 (+ 136us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.054741 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.054893 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260620 23:06:18.055606 32267 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45551
I20260620 23:06:18.055707 32267 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:18.055936 32267 heartbeater.cc:511] Master 127.17.188.126:45551 requested a full tablet report, sending...
I20260620 23:06:18.056329 31942 ts_manager.cc:194] Registered new tserver with Master: fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871)
I20260620 23:06:18.056910 31942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:44103
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:06:18.074270 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45551
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:18.151765 32272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:18.151953 32272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:18.151975 32272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:18.153041 32272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:18.153075 32272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:18.153113 32272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:06:18.154480 32272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:18.154656 32272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:18.154812 32272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:18.159511 32277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:18.159559 32272 server_base.cc:1034] running on GCE node
W20260620 23:06:18.159680 32278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:18.159595 32280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:18.159933 32272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:18.160161 32272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:18.161361 32272 hybrid_clock.cc:648] HybridClock initialized: now 1781996778161341 us; error 38 us; skew 500 ppm
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:18.163584 32272 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:06:18.164014 32272 webserver.cc:460] Webserver started at http://127.17.188.67:46509/ using document root <none> and password file <none>
I20260620 23:06:18.164234 32272 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:18.164275 32272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:18.164407 32272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:18.165493 32272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "bcd4e21fe26f40fca224dec54e0004c8"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "282ba908e0d279a3efa7524def376fb0"
server_key_iv: "5508ffa891ee47853d88b50a2ec45e1d"
server_key_version: "encryptionkey@0"
I20260620 23:06:18.165884 32272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "bcd4e21fe26f40fca224dec54e0004c8"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "282ba908e0d279a3efa7524def376fb0"
server_key_iv: "5508ffa891ee47853d88b50a2ec45e1d"
server_key_version: "encryptionkey@0"
I20260620 23:06:18.167151 32272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:06:18.167832 32287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.168010 32272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:18.168102 32272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "bcd4e21fe26f40fca224dec54e0004c8"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "282ba908e0d279a3efa7524def376fb0"
server_key_iv: "5508ffa891ee47853d88b50a2ec45e1d"
server_key_version: "encryptionkey@0"
I20260620 23:06:18.168174 32272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:18.181733 32272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:18.183391 32272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:18.183534 32272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:18.183749 32272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:18.184049 32272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:18.184087 32272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.184108 32272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:18.184154 32272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.190265 32272 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:39295
I20260620 23:06:18.190315 32400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:39295 every 8 connection(s)
I20260620 23:06:18.190619 32272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:18.198501 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 32272
I20260620 23:06:18.198632 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:06:18.198802 18161 external_mini_cluster.cc:1428] Setting key 02018322caf85389c58d7867c51d459a
I20260620 23:06:18.199842 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.191390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:52321 (local address 127.17.188.126:45551)
0620 23:06:18.191431 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.191433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.191672 (+ 239us) server_negotiation.cc:407] Connection header received
0620 23:06:18.192200 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.192202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.192215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.192267 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:18.193216 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.193765 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.194773 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.194971 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.195062 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.197011 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:18.197012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:18.197012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:18.197019 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:18.198404 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.198827 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.198828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.198828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.198893 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.199166 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.199166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.199167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.199315 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:18.199398 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.199711 (+ 313us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.199783 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:06:18.200031 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.191359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:52321)
0620 23:06:18.191555 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.191589 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.192087 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.192333 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.192336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.192930 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:18.193109 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.193113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.193871 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.193872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.194663 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.194666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.195072 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.195073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.195194 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.195629 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:18.195636 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:18.196929 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:18.198486 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.198491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.198492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.198757 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.198963 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.198964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.198965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.199096 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.199399 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:18.199406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:18.199565 (+ 159us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.199675 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.199808 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260620 23:06:18.200476 32401 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45551
I20260620 23:06:18.200546 32401 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:18.200668 32401 heartbeater.cc:511] Master 127.17.188.126:45551 requested a full tablet report, sending...
I20260620 23:06:18.200996 31942 ts_manager.cc:194] Registered new tserver with Master: bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295)
I20260620 23:06:18.201298 31942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:52321
I20260620 23:06:18.210299 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:06:18.216975 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.211609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58562 (local address 127.17.188.126:45551)
0620 23:06:18.211660 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.211662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.211732 (+ 70us) server_negotiation.cc:407] Connection header received
0620 23:06:18.211770 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.211770 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.211778 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.211816 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:18.212329 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.212849 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.213574 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.213745 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.213795 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.214931 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:18.214932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:18.214932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:18.214938 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:18.216103 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.216389 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.216389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.216390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.216434 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.216588 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.216589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.216589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.216722 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:18.216793 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.216868 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.216920 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:06:18.218405 31942 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:06:18.219875 31942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:06:18.225965 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.221827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:39295 (local address 127.0.0.1:33126)
0620 23:06:18.222000 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.222040 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.222102 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.222651 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.222652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.222653 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:18.222716 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.222718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.223532 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.223533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.224692 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.224694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.225664 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.225665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.225755 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.225767 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.225846 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.225897 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260620 23:06:18.226007 32412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.221962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.17.188.66:33871)
0620 23:06:18.222147 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.222152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.222190 (+ 38us) server_negotiation.cc:407] Connection header received
0620 23:06:18.222323 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.222326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.222363 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.222447 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:18.222716 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.223820 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.225040 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.225590 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.225684 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.225749 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.225891 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.225934 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260620 23:06:18.226007 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.221926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33126 (local address 127.17.188.67:39295)
0620 23:06:18.222458 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.222464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.222473 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:06:18.222509 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.222511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.222525 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.222595 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:18.222777 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.223434 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.224778 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.225515 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.225579 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.225630 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.225890 (+ 260us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.225932 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":77,"threads_started":1}
I20260620 23:06:18.226241 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.221853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:33871 (local address 127.0.0.1:35662)
0620 23:06:18.222129 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.222139 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.222221 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.222553 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.222556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.222558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:18.222616 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.222617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.223929 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.223930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.224951 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.224952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.225689 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.225690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.225764 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.225769 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.226111 (+ 342us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.226156 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":50,"threads_started":1}
I20260620 23:06:18.226362 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.222049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36745 (local address 127.0.0.1:49002)
0620 23:06:18.222187 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.222196 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.222238 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.222672 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.222673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.222675 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:18.222742 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.222744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.223973 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.223974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.224961 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.224963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.225928 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.225929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.225997 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.226001 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.226233 (+ 232us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.226297 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":19,"thread_start_us":34,"threads_started":1}
I20260620 23:06:18.226867 32335 tablet_service.cc:1467] Processing CreateTablet for tablet a06813c696e54b0a9ce49f370297427e (DEFAULT_TABLE table=test-table [id=ce47b1b4739d46839c5846e7102d55cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:18.226953 32201 tablet_service.cc:1467] Processing CreateTablet for tablet a06813c696e54b0a9ce49f370297427e (DEFAULT_TABLE table=test-table [id=ce47b1b4739d46839c5846e7102d55cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:18.227213 32335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a06813c696e54b0a9ce49f370297427e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:18.227254 32201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a06813c696e54b0a9ce49f370297427e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:18.229738 32414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.222227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49002 (local address 127.17.188.65:36745)
0620 23:06:18.222440 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.222444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.222454 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:06:18.222495 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.222498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.222514 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.222589 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:18.222826 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.223866 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.225050 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.225795 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.225853 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.225898 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.229614 (+ 3716us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.229657 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":112,"threads_started":1}
I20260620 23:06:18.229734 32416 tablet_bootstrap.cc:492] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: Bootstrap starting.
I20260620 23:06:18.230098 32415 tablet_bootstrap.cc:492] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: Bootstrap starting.
I20260620 23:06:18.230540 32416 tablet_bootstrap.cc:654] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:18.230391 32067 tablet_service.cc:1467] Processing CreateTablet for tablet a06813c696e54b0a9ce49f370297427e (DEFAULT_TABLE table=test-table [id=ce47b1b4739d46839c5846e7102d55cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:18.230664 32415 tablet_bootstrap.cc:654] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:18.230765 32067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a06813c696e54b0a9ce49f370297427e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:18.231117 32416 log.cc:824] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:18.231145 32415 log.cc:824] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:18.231877 32416 tablet_bootstrap.cc:492] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: No bootstrap required, opened a new log
I20260620 23:06:18.231948 32416 ts_tablet_manager.cc:1397] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:06:18.231993 32415 tablet_bootstrap.cc:492] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: No bootstrap required, opened a new log
I20260620 23:06:18.232060 32415 ts_tablet_manager.cc:1397] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:06:18.233557 32415 raft_consensus.cc:348] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.233708 32415 raft_consensus.cc:374] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:18.233752 32415 raft_consensus.cc:729] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd4e21fe26f40fca224dec54e0004c8, State: Initialized, Role: FOLLOWER
I20260620 23:06:18.233860 32415 consensus_queue.cc:260] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.234148 32415 ts_tablet_manager.cc:1428] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:06:18.234066 32416 raft_consensus.cc:348] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.234207 32401 heartbeater.cc:503] Master 127.17.188.126:45551 was elected leader, sending a full tablet report...
I20260620 23:06:18.234212 32416 raft_consensus.cc:374] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:18.234251 32416 raft_consensus.cc:729] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7ab3cfb5564290ada93f93290bc40d, State: Initialized, Role: FOLLOWER
I20260620 23:06:18.234365 32416 consensus_queue.cc:260] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.234653 32416 ts_tablet_manager.cc:1428] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:06:18.234690 32267 heartbeater.cc:503] Master 127.17.188.126:45551 was elected leader, sending a full tablet report...
I20260620 23:06:18.234825 32419 tablet_bootstrap.cc:492] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Bootstrap starting.
I20260620 23:06:18.235576 32419 tablet_bootstrap.cc:654] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:18.236022 32419 log.cc:824] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:18.236685 32419 tablet_bootstrap.cc:492] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: No bootstrap required, opened a new log
I20260620 23:06:18.236763 32419 ts_tablet_manager.cc:1397] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:06:18.238104 32419 raft_consensus.cc:348] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.238251 32419 raft_consensus.cc:374] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:18.238296 32419 raft_consensus.cc:729] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378924fa0b92429fa1e0576446721836, State: Initialized, Role: FOLLOWER
I20260620 23:06:18.238423 32419 consensus_queue.cc:260] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.238677 32419 ts_tablet_manager.cc:1428] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:06:18.238720 32133 heartbeater.cc:503] Master 127.17.188.126:45551 was elected leader, sending a full tablet report...
W20260620 23:06:18.296507 32268 tablet.cc:2369] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:06:18.390213 32134 tablet.cc:2369] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:06:18.441413 32402 tablet.cc:2369] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:06:18.458952 32420 raft_consensus.cc:482] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:18.459061 32420 raft_consensus.cc:504] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.459460 32420 leader_election.cc:288] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871), 378924fa0b92429fa1e0576446721836 (127.17.188.65:36745)
I20260620 23:06:18.463069 32412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.459719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:49021 (local address 127.17.188.66:33871)
0620 23:06:18.459783 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.459786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.459825 (+ 39us) server_negotiation.cc:407] Connection header received
0620 23:06:18.459849 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.459850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.459857 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.459923 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:18.460103 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.460838 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.462145 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.462720 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.462795 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.462803 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.462957 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.462992 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:06:18.463078 32414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.459917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:55291 (local address 127.17.188.65:36745)
0620 23:06:18.459965 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.459966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.460009 (+ 43us) server_negotiation.cc:407] Connection header received
0620 23:06:18.460095 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.460096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.460105 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.460160 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:18.460373 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.461016 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.462193 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.462720 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.462794 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.462803 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.462956 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.462992 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 23:06:18.463100 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.459651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:33871 (local address 127.17.188.67:49021)
0620 23:06:18.459707 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.459718 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.459764 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.459952 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.459953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.459955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:18.460026 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.460028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.460914 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.460915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.462053 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.462056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.462800 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.462801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.462888 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.462890 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.462939 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.462980 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":9}
I20260620 23:06:18.463106 32424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.459792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36745 (local address 127.17.188.67:55291)
0620 23:06:18.459952 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.459962 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.460014 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.460170 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.460171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.460203 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:18.460281 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.460282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.461087 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.461087 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.462125 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.462128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.462800 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.462801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.462871 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.462874 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.462945 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.462980 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260620 23:06:18.463464 32087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06813c696e54b0a9ce49f370297427e" candidate_uuid: "bcd4e21fe26f40fca224dec54e0004c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378924fa0b92429fa1e0576446721836" is_pre_election: true
I20260620 23:06:18.463465 32221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06813c696e54b0a9ce49f370297427e" candidate_uuid: "bcd4e21fe26f40fca224dec54e0004c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb7ab3cfb5564290ada93f93290bc40d" is_pre_election: true
I20260620 23:06:18.463649 32087 raft_consensus.cc:2454] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcd4e21fe26f40fca224dec54e0004c8 in term 0.
I20260620 23:06:18.463656 32221 raft_consensus.cc:2454] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcd4e21fe26f40fca224dec54e0004c8 in term 0.
I20260620 23:06:18.463899 32290 leader_election.cc:302] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcd4e21fe26f40fca224dec54e0004c8, fb7ab3cfb5564290ada93f93290bc40d; no voters:
I20260620 23:06:18.464061 32420 raft_consensus.cc:2781] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:06:18.464131 32420 raft_consensus.cc:482] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:06:18.464164 32420 raft_consensus.cc:3037] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:18.464871 32420 raft_consensus.cc:504] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.465004 32420 leader_election.cc:288] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [CANDIDATE]: Term 1 election: Requested vote from peers fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871), 378924fa0b92429fa1e0576446721836 (127.17.188.65:36745)
I20260620 23:06:18.465168 32221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06813c696e54b0a9ce49f370297427e" candidate_uuid: "bcd4e21fe26f40fca224dec54e0004c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb7ab3cfb5564290ada93f93290bc40d"
I20260620 23:06:18.465171 32087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06813c696e54b0a9ce49f370297427e" candidate_uuid: "bcd4e21fe26f40fca224dec54e0004c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378924fa0b92429fa1e0576446721836"
I20260620 23:06:18.465232 32221 raft_consensus.cc:3037] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:18.465260 32087 raft_consensus.cc:3037] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:18.465939 32221 raft_consensus.cc:2454] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcd4e21fe26f40fca224dec54e0004c8 in term 1.
I20260620 23:06:18.465968 32087 raft_consensus.cc:2454] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcd4e21fe26f40fca224dec54e0004c8 in term 1.
I20260620 23:06:18.466125 32290 leader_election.cc:302] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378924fa0b92429fa1e0576446721836, bcd4e21fe26f40fca224dec54e0004c8; no voters:
I20260620 23:06:18.466226 32420 raft_consensus.cc:2781] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:18.466431 32420 raft_consensus.cc:686] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 1 LEADER]: Becoming Leader. State: Replica: bcd4e21fe26f40fca224dec54e0004c8, State: Running, Role: LEADER
I20260620 23:06:18.466535 32420 consensus_queue.cc:237] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.467314 31942 catalog_manager.cc:5495] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67). New cstate: current_term: 1 leader_uuid: "bcd4e21fe26f40fca224dec54e0004c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } health_report { overall_health: UNKNOWN } } }
I20260620 23:06:18.506482 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.504008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.17.188.67:39295)
0620 23:06:18.504080 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.504081 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.504089 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:06:18.504127 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.504130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.504152 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.504230 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:06:18.504367 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.504873 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.505656 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.505869 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.505941 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.506124 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:06:18.506230 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:06:18.506318 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.506359 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.506380 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 23:06:18.508291 32087 raft_consensus.cc:1261] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Refusing update from remote peer bcd4e21fe26f40fca224dec54e0004c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:18.508337 32221 raft_consensus.cc:1261] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [term 1 FOLLOWER]: Refusing update from remote peer bcd4e21fe26f40fca224dec54e0004c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:18.508663 32420 consensus_queue.cc:1035] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:18.508826 32425 consensus_queue.cc:1035] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:18.511490 32429 mvcc.cc:205] Tried to move back new op lower bound from 7299058804767432704 to 7299058804599562240. Current Snapshot: MvccSnapshot[applied={T|T < 7299058804767432704}]
I20260620 23:06:18.512271 32430 mvcc.cc:205] Tried to move back new op lower bound from 7299058804767432704 to 7299058804599562240. Current Snapshot: MvccSnapshot[applied={T|T < 7299058804767432704}]
I20260620 23:06:18.513666 31942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:18.513748 31942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:18.514477 31942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Sending DeleteTablet for 3 replicas of tablet a06813c696e54b0a9ce49f370297427e
I20260620 23:06:18.514791 32335 tablet_service.cc:1514] Processing DeleteTablet for tablet a06813c696e54b0a9ce49f370297427e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:18 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:33126
I20260620 23:06:18.514912 32067 tablet_service.cc:1514] Processing DeleteTablet for tablet a06813c696e54b0a9ce49f370297427e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:18 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:49002
I20260620 23:06:18.515024 32437 tablet_replica.cc:330] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: stopping tablet replica
I20260620 23:06:18.515127 32437 raft_consensus.cc:2229] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:06:18.515130 32201 tablet_service.cc:1514] Processing DeleteTablet for tablet a06813c696e54b0a9ce49f370297427e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:18 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:35662
I20260620 23:06:18.515233 32437 raft_consensus.cc:2258] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:18.515226 32438 tablet_replica.cc:330] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: stopping tablet replica
I20260620 23:06:18.515322 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 32004
I20260620 23:06:18.516723 32439 tablet_replica.cc:330] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: stopping tablet replica
I20260620 23:06:18.516827 32439 raft_consensus.cc:2229] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:06:18.516925 32439 raft_consensus.cc:2258] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:18.517361 32439 ts_tablet_manager.cc:1905] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:18.518805 32439 ts_tablet_manager.cc:1918] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:18.518895 32439 log.cc:1196] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/a06813c696e54b0a9ce49f370297427e
I20260620 23:06:18.519147 32439 ts_tablet_manager.cc:1939] T a06813c696e54b0a9ce49f370297427e P fb7ab3cfb5564290ada93f93290bc40d: Deleting consensus metadata
I20260620 23:06:18.519526 31928 catalog_manager.cc:4841] TS fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871): tablet a06813c696e54b0a9ce49f370297427e (table test-table [id=ce47b1b4739d46839c5846e7102d55cf]) successfully deleted
W20260620 23:06:18.521554 31928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:36745 (error 108)
I20260620 23:06:18.521756 32437 ts_tablet_manager.cc:1905] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:18.521802 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 32138
I20260620 23:06:18.524255 32437 ts_tablet_manager.cc:1918] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:18.524327 32437 log.cc:1196] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/a06813c696e54b0a9ce49f370297427e
I20260620 23:06:18.524626 32437 ts_tablet_manager.cc:1939] T a06813c696e54b0a9ce49f370297427e P bcd4e21fe26f40fca224dec54e0004c8: Deleting consensus metadata
I20260620 23:06:18.525018 31927 catalog_manager.cc:4841] TS bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295): tablet a06813c696e54b0a9ce49f370297427e (table test-table [id=ce47b1b4739d46839c5846e7102d55cf]) successfully deleted
I20260620 23:06:18.525727 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.525584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36745 (local address 127.0.0.1:49014)
0620 23:06:18.525634 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.525675 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:36745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 23:06:18.525840 31928 catalog_manager.cc:4584] TS 378924fa0b92429fa1e0576446721836 (127.17.188.65:36745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a06813c696e54b0a9ce49f370297427e: Network error: Client connection negotiation failed: client connection to 127.17.188.65:36745: connect: Connection refused (error 111)
I20260620 23:06:18.528349 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 32272
I20260620 23:06:18.532855 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 31910
I20260620 23:06:18.538517 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45551
--webserver_interface=127.17.188.126
--webserver_port=42067
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:45551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:18.618010 32441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:18.618230 32441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:18.618253 32441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:18.619405 32441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:18.619446 32441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:18.619462 32441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:18.619474 32441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:18.619488 32441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:18.620808 32441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:45551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:18.620996 32441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:18.621232 32441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:18.629566 32450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:18.629566 32447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:18.629573 32448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:18.629594 32441 server_base.cc:1034] running on GCE node
I20260620 23:06:18.629963 32441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:18.630265 32441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:18.631404 32441 hybrid_clock.cc:648] HybridClock initialized: now 1781996778631395 us; error 26 us; skew 500 ppm
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:18.633816 32441 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:06:18.634292 32441 webserver.cc:460] Webserver started at http://127.17.188.126:42067/ using document root <none> and password file <none>
I20260620 23:06:18.634522 32441 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:18.634603 32441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:18.635847 32441 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:18.636364 32457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.636529 32441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.636610 32441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "523c075900a947dc9016a4330d826d6e"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "d17e8788e00d2e477a0a5f8552a38124"
server_key_iv: "4ac0978911ef0673053c9fa561038e88"
server_key_version: "encryptionkey@0"
I20260620 23:06:18.636914 32441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:18.650918 32441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:18.652659 32441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:18.652763 32441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:18.656329 32441 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:45551
I20260620 23:06:18.656401 32509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:45551 every 8 connection(s)
I20260620 23:06:18.656737 32441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:06:18.658210 32510 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:06:18.658823 32510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Bootstrap starting.
I20260620 23:06:18.662858 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 32441
I20260620 23:06:18.663170 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:36745
--local_ip_for_outbound_sockets=127.17.188.65
--tserver_master_addrs=127.17.188.126:45551
--webserver_port=41487
--webserver_interface=127.17.188.65
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:18.667084 32510 log.cc:824] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:18.668668 32510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:06:18.668923 32510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Bootstrap complete.
I20260620 23:06:18.671046 32510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:18.671280 32510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 523c075900a947dc9016a4330d826d6e, State: Initialized, Role: FOLLOWER
I20260620 23:06:18.671455 32510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:18.671533 32510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:06:18.671586 32510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:06:18.671635 32510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:06:18.672518 32510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:18.672622 32510 leader_election.cc:302] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 523c075900a947dc9016a4330d826d6e; no voters:
I20260620 23:06:18.672792 32510 leader_election.cc:288] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:06:18.672822 32514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:06:18.672927 32514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 2 LEADER]: Becoming Leader. State: Replica: 523c075900a947dc9016a4330d826d6e, State: Running, Role: LEADER
I20260620 23:06:18.673048 32510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:18.673038 32514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:18.673291 32514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 523c075900a947dc9016a4330d826d6e. Latest consensus state: current_term: 2 leader_uuid: "523c075900a947dc9016a4330d826d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } } }
I20260620 23:06:18.673373 32514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:18.673508 32515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "523c075900a947dc9016a4330d826d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } } }
I20260620 23:06:18.673573 32515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:18.673851 32518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:06:18.674576 32518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:06:18.674677 32518 catalog_manager.cc:1259] Loaded cluster ID: 31c4c220998a4f40a7b859d743c4eb3e
I20260620 23:06:18.674714 32518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:06:18.674934 32518 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 23:06:18.679693 32518 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 23:06:18.682092 32518 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:06:18.682276 32518 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Loaded TSK: 0
I20260620 23:06:18.682500 32518 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:06:18.744447 32512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:18.744683 32512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:18.744721 32512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:18.745858 32512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:18.745920 32512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:18.745966 32512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:06:18.747386 32512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:36745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=41487
--tserver_master_addrs=127.17.188.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:18.747674 32512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:18.747889 32512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:18.753213 32538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:18.753227 32540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:18.753352 32537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:18.753545 32512 server_base.cc:1034] running on GCE node
I20260620 23:06:18.753733 32512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:18.753988 32512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:18.755170 32512 hybrid_clock.cc:648] HybridClock initialized: now 1781996778755156 us; error 26 us; skew 500 ppm
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:18.758054 32512 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:06:18.758555 32512 webserver.cc:460] Webserver started at http://127.17.188.65:41487/ using document root <none> and password file <none>
I20260620 23:06:18.758809 32512 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:18.758886 32512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:18.760511 32512 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:18.761232 32547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.761417 32512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:18.761505 32512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "378924fa0b92429fa1e0576446721836"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "c0b595daed0ff6d31b401d46b6a42dd6"
server_key_iv: "214545342f66ab82a62eacda668dfb61"
server_key_version: "encryptionkey@0"
I20260620 23:06:18.761888 32512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:18.790064 32512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:18.792304 32512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:18.792459 32512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:18.792675 32512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:18.793104 32554 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 23:06:18.794540 32512 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 23:06:18.794595 32512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 23:06:18.794625 32512 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 23:06:18.795400 32512 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 23:06:18.795444 32512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 23:06:18.795523 32554 tablet_bootstrap.cc:492] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Bootstrap starting.
I20260620 23:06:18.801967 32512 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:36745
I20260620 23:06:18.802187 32661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:36745 every 8 connection(s)
I20260620 23:06:18.802333 32512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:06:18.805462 32554 log.cc:824] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:18.807778 32554 tablet_bootstrap.cc:492] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:06:18.808069 32554 tablet_bootstrap.cc:492] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Bootstrap complete.
I20260620 23:06:18.808521 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 32512
I20260620 23:06:18.808794 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:33871
--local_ip_for_outbound_sockets=127.17.188.66
--tserver_master_addrs=127.17.188.126:45551
--webserver_port=37211
--webserver_interface=127.17.188.66
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:18.809218 32554 ts_tablet_manager.cc:1397] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.006s
I20260620 23:06:18.810534 32554 raft_consensus.cc:348] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.810849 32554 raft_consensus.cc:729] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378924fa0b92429fa1e0576446721836, State: Initialized, Role: FOLLOWER
I20260620 23:06:18.811009 32554 consensus_queue.cc:260] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:18.811273 32554 ts_tablet_manager.cc:1428] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:18.816715 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.803130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:56105 (local address 127.17.188.126:45551)
0620 23:06:18.805481 (+ 2351us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.805491 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.805731 (+ 240us) server_negotiation.cc:407] Connection header received
0620 23:06:18.806263 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.806271 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.806330 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.806487 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:18.807205 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.808567 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.810108 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.810693 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.810792 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.813270 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:18.813274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:18.813274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:18.813285 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:18.815006 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.815517 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.815520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.815520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.815603 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.815911 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.815911 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.815911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.816100 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:18.816157 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.816496 (+ 339us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.816589 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2299,"thread_start_us":92,"threads_started":1}
I20260620 23:06:18.816865 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.803045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:56105)
0620 23:06:18.805439 (+ 2394us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.805484 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.806056 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.806526 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.806530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.806843 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:18.807083 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.807088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.808689 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.808690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.810010 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.810012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.810880 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.810881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.811017 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.811674 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:18.811682 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:18.813161 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:18.815092 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.815100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.815102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.815434 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.815675 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.815676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.815676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.815833 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.816165 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:18.816171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:18.816332 (+ 161us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.816457 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.816629 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2352,"thread_start_us":89,"threads_started":1}
I20260620 23:06:18.817584 32662 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45551
I20260620 23:06:18.817682 32662 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:18.817836 32662 heartbeater.cc:511] Master 127.17.188.126:45551 requested a full tablet report, sending...
I20260620 23:06:18.818598 32474 ts_manager.cc:194] Registered new tserver with Master: 378924fa0b92429fa1e0576446721836 (127.17.188.65:36745)
I20260620 23:06:18.819851 32474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:56105
I20260620 23:06:18.820644 32662 heartbeater.cc:503] Master 127.17.188.126:45551 was elected leader, sending a full tablet report...
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:06:18.827337 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.819243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36745 (local address 127.0.0.1:49018)
0620 23:06:18.819464 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.819486 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.819535 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.819747 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.819752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.820050 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:18.820093 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.820095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.821436 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.821437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.821959 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.821961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.822593 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.822594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.822640 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.823044 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:18.823047 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:18.824234 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:18.826062 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.826066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.826068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.826276 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.826484 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.826485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.826485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.826551 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.826920 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:18.826926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:18.826984 (+ 58us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.827083 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.827208 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":87,"threads_started":1}
I20260620 23:06:18.827364 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.819287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49018 (local address 127.17.188.65:36745)
0620 23:06:18.819620 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.819624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.819630 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:06:18.819652 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.819653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.819662 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.819734 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:18.820179 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.821335 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.822040 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.822511 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.822591 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.824309 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:18.824316 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:18.824316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:18.824323 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:18.825960 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.826346 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.826349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.826349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.826406 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.826616 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.826617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.826617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.826818 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:18.826923 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.827121 (+ 198us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.827254 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":55,"threads_started":1}
I20260620 23:06:18.828123 32596 tablet_service.cc:1514] Processing DeleteTablet for tablet a06813c696e54b0a9ce49f370297427e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:18 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:49018
I20260620 23:06:18.828665 32671 tablet_replica.cc:330] stopping tablet replica
I20260620 23:06:18.828759 32671 raft_consensus.cc:2229] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:06:18.828821 32671 raft_consensus.cc:2258] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:18.829119 32671 ts_tablet_manager.cc:1905] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:18.830983 32671 ts_tablet_manager.cc:1918] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:18.831037 32671 log.cc:1196] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/a06813c696e54b0a9ce49f370297427e
I20260620 23:06:18.831234 32671 ts_tablet_manager.cc:1939] T a06813c696e54b0a9ce49f370297427e P 378924fa0b92429fa1e0576446721836: Deleting consensus metadata
I20260620 23:06:18.831622 32460 catalog_manager.cc:4841] TS 378924fa0b92429fa1e0576446721836 (127.17.188.65:36745): tablet a06813c696e54b0a9ce49f370297427e (table test-table [id=ce47b1b4739d46839c5846e7102d55cf]) successfully deleted
W20260620 23:06:18.892475 32667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:18.892701 32667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:18.892738 32667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:18.893884 32667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:18.893946 32667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:18.893986 32667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:06:18.895398 32667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:33871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=37211
--tserver_master_addrs=127.17.188.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:18.895677 32667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:18.895923 32667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:18.900557 32679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:18.900571 32667 server_base.cc:1034] running on GCE node
W20260620 23:06:18.900580 32676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:18.900558 32677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:18.900914 32667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:18.901154 32667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:18.902309 32667 hybrid_clock.cc:648] HybridClock initialized: now 1781996778902301 us; error 37 us; skew 500 ppm
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:18.904610 32667 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:06:18.904959 32667 webserver.cc:460] Webserver started at http://127.17.188.66:37211/ using document root <none> and password file <none>
I20260620 23:06:18.905135 32667 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:18.905182 32667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:18.906463 32667 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:18.906935 32686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.907114 32667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.907217 32667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "fb7ab3cfb5564290ada93f93290bc40d"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "85cc28e43ca350eefe31cba61b771711"
server_key_iv: "e628661e2714710c30acf3cf565059f6"
server_key_version: "encryptionkey@0"
I20260620 23:06:18.907495 32667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:18.924628 32667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:18.926352 32667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:18.926501 32667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:18.926690 32667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:18.926963 32667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:18.926994 32667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.927014 32667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:18.927054 32667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:18.932617 32667 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:33871
I20260620 23:06:18.932669 331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:33871 every 8 connection(s)
I20260620 23:06:18.932978 32667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:06:18.936658 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 32667
I20260620 23:06:18.936940 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:39295
--local_ip_for_outbound_sockets=127.17.188.67
--tserver_master_addrs=127.17.188.126:45551
--webserver_port=46509
--webserver_interface=127.17.188.67
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:06:18 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996778, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:18.942695 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.933577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:54783)
0620 23:06:18.933759 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:18.933821 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:18.934294 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:18.934527 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:18.934531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:18.935049 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:18.935222 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.935227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.935911 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.935912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.936689 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:18.936692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.937133 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.937134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:18.937245 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.937767 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:18.937775 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:18.939297 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:18.941279 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.941286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.941287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.941580 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.941832 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:18.941833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:18.941833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:18.941931 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:18.942232 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:18.942236 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:18.942345 (+ 109us) client_negotiation.cc:769] Sending connection context
0620 23:06:18.942429 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 23:06:18.942546 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260620 23:06:18.942706 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:18.933618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:54783 (local address 127.17.188.126:45551)
0620 23:06:18.933686 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:18.933687 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:18.933905 (+ 218us) server_negotiation.cc:407] Connection header received
0620 23:06:18.934396 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:18.934398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:18.934409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:18.934450 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:18.935319 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.935821 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.936789 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:18.936997 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:18.937068 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:18.939381 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:18.939383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:18.939383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:18.939391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:18.941179 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.941692 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.941693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.941693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.941757 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:18.942005 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:18.942006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:18.942007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:18.942158 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:18.942237 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:18.942473 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 23:06:18.942571 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 23:06:18.943353 332 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45551
I20260620 23:06:18.943457 332 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:18.943657 332 heartbeater.cc:511] Master 127.17.188.126:45551 requested a full tablet report, sending...
I20260620 23:06:18.944073 32474 ts_manager.cc:194] Registered new tserver with Master: fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871)
I20260620 23:06:18.944444 32474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:54783
W20260620 23:06:19.015913 335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:19.016093 335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:19.016115 335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:19.017251 335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:19.017290 335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:19.017378 335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:06:19.018745 335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:39295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=46509
--tserver_master_addrs=127.17.188.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:19.018949 335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:19.019147 335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:19.023809 343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:19.023836 340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:19.023883 335 server_base.cc:1034] running on GCE node
W20260620 23:06:19.023849 341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:19.024175 335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:19.024389 335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:19.025518 335 hybrid_clock.cc:648] HybridClock initialized: now 1781996779025518 us; error 38 us; skew 500 ppm
Jun 20 23:06:19 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:19.027858 335 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:06:19.028285 335 webserver.cc:460] Webserver started at http://127.17.188.67:46509/ using document root <none> and password file <none>
I20260620 23:06:19.028532 335 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:19.028599 335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:19.029904 335 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:19.030435 350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.030586 335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:19.030674 335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "bcd4e21fe26f40fca224dec54e0004c8"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "282ba908e0d279a3efa7524def376fb0"
server_key_iv: "5508ffa891ee47853d88b50a2ec45e1d"
server_key_version: "encryptionkey@0"
I20260620 23:06:19.030954 335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:19.042296 335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:19.043926 335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:19.044078 335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:19.044286 335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:19.044585 335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:19.044637 335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.044677 335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:19.044705 335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.050061 335 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:39295
I20260620 23:06:19.050094 463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:39295 every 8 connection(s)
I20260620 23:06:19.050405 335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:06:19.051481 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 335
Jun 20 23:06:19 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:19.058547 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.051864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58566 (local address 127.17.188.126:45551)
0620 23:06:19.051993 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:19.051995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:19.052051 (+ 56us) server_negotiation.cc:407] Connection header received
0620 23:06:19.052120 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:19.052120 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:19.052132 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:19.052182 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:19.052722 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.053212 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.053932 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.054142 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.054223 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.055514 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:19.055515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:19.055516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:19.055523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:19.057303 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:19.057707 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:19.057708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:19.057709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:19.057771 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:19.057964 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:19.057965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:19.057966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:19.058113 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:19.058289 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:19.058329 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 23:06:19.058401 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260620 23:06:19.059700 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.051052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:55623 (local address 127.17.188.126:45551)
0620 23:06:19.051104 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:19.051106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:19.051250 (+ 144us) server_negotiation.cc:407] Connection header received
0620 23:06:19.051735 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:19.051736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:19.051767 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:19.051828 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:19.052768 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.053294 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.054257 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.054442 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.054558 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.056657 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:19.056658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:19.056659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:19.056665 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:19.058285 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:19.058801 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:19.058802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:19.058803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:19.058855 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:19.059150 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:19.059152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:19.059152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:19.059258 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:19.059338 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:19.059596 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 23:06:19.059649 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 23:06:19.059830 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.051042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:55623)
0620 23:06:19.051163 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.051192 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:19.051651 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:19.051969 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:19.051973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:19.052504 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:19.052666 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.052670 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.053419 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.053419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.054156 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.054158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.054557 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.054558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.054687 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.055120 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:19.055127 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:19.056552 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:19.058365 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:19.058371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:19.058373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:19.058657 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:19.058940 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:19.058941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:19.058941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:19.059070 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:19.059333 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:19.059338 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:19.059487 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 23:06:19.059568 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 23:06:19.059671 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":70,"threads_started":1}
I20260620 23:06:19.060274 464 heartbeater.cc:347] Connected to a master server at 127.17.188.126:45551
I20260620 23:06:19.060356 464 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:19.060525 464 heartbeater.cc:511] Master 127.17.188.126:45551 requested a full tablet report, sending...
I20260620 23:06:19.060874 32474 ts_manager.cc:194] Registered new tserver with Master: bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295)
I20260620 23:06:19.061179 32474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:55623
I20260620 23:06:19.070137 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:06:19.072304 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.070415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58576 (local address 127.17.188.126:45551)
0620 23:06:19.070465 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:19.070467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:19.070576 (+ 109us) server_negotiation.cc:407] Connection header received
0620 23:06:19.070609 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:19.070609 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:19.070644 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:19.070693 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:06:19.070778 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.071154 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.071741 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.071889 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.071931 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.072046 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:06:19.072116 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:06:19.072185 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:19.072218 (+ 33us) server_negotiation.cc:299] Negotiation successful
0620 23:06:19.072230 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 23:06:19.072532 32474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:06:19.074250 32474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:06:19.076308 32596 tablet_service.cc:1467] Processing CreateTablet for tablet 88178fa48ab846e1a3b3d7eee25cfd8f (DEFAULT_TABLE table=test-table [id=6546496b82024090b65d4f0d0f1cc453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:19.076489 32596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88178fa48ab846e1a3b3d7eee25cfd8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:19.077980 32554 tablet_bootstrap.cc:492] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Bootstrap starting.
I20260620 23:06:19.078322 32554 tablet_bootstrap.cc:654] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:19.078857 32554 tablet_bootstrap.cc:492] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: No bootstrap required, opened a new log
I20260620 23:06:19.078910 32554 ts_tablet_manager.cc:1397] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 23:06:19.079299 32554 raft_consensus.cc:348] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.079380 32554 raft_consensus.cc:374] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:19.079416 32554 raft_consensus.cc:729] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378924fa0b92429fa1e0576446721836, State: Initialized, Role: FOLLOWER
I20260620 23:06:19.079464 32554 consensus_queue.cc:260] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.079551 32554 ts_tablet_manager.cc:1428] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 23:06:19.080154 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.075902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:33871 (local address 127.0.0.1:35668)
0620 23:06:19.076045 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.076054 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:19.076096 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:19.076450 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:19.076451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:19.076453 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:19.076484 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.076485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.077563 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.077564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.078756 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.078758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.079893 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.079894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.079960 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.079975 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 23:06:19.080034 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 23:06:19.080072 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":37,"thread_start_us":98,"threads_started":1}
I20260620 23:06:19.080415 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.076063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.17.188.66:33871)
0620 23:06:19.076241 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:19.076245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:19.076252 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:06:19.076282 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:19.076283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:19.076294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:19.076385 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:19.076535 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.077454 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.079006 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.079818 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.079871 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.079907 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:19.080284 (+ 377us) server_negotiation.cc:299] Negotiation successful
0620 23:06:19.080334 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":80,"threads_started":1}
I20260620 23:06:19.080960 471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.075922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.17.188.67:39295)
0620 23:06:19.076203 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:19.076208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:19.076217 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:06:19.076250 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:19.076252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:19.076265 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:19.076341 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:19.076476 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.077578 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.078846 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.080634 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.080733 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.080783 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:19.080883 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 23:06:19.080908 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":81,"threads_started":1}
I20260620 23:06:19.080977 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.075873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:39295 (local address 127.0.0.1:33146)
0620 23:06:19.075922 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.075931 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:19.075970 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:19.076362 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:19.076363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:19.076365 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:19.076407 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.076408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.077664 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.077665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.078756 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.078758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.080738 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.080739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.080808 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.080812 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:06:19.080876 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:06:19.080915 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":4}
I20260620 23:06:19.081149 32734 tablet_service.cc:1467] Processing CreateTablet for tablet 88178fa48ab846e1a3b3d7eee25cfd8f (DEFAULT_TABLE table=test-table [id=6546496b82024090b65d4f0d0f1cc453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:19.081516 32734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88178fa48ab846e1a3b3d7eee25cfd8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:19.081660 398 tablet_service.cc:1467] Processing CreateTablet for tablet 88178fa48ab846e1a3b3d7eee25cfd8f (DEFAULT_TABLE table=test-table [id=6546496b82024090b65d4f0d0f1cc453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:19.081935 398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88178fa48ab846e1a3b3d7eee25cfd8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:19.083992 474 tablet_bootstrap.cc:492] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: Bootstrap starting.
I20260620 23:06:19.084379 475 tablet_bootstrap.cc:492] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: Bootstrap starting.
I20260620 23:06:19.084695 474 tablet_bootstrap.cc:654] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:19.084980 475 tablet_bootstrap.cc:654] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:19.085214 474 log.cc:824] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:19.085582 475 log.cc:824] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:19.086038 474 tablet_bootstrap.cc:492] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: No bootstrap required, opened a new log
I20260620 23:06:19.086105 474 ts_tablet_manager.cc:1397] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:06:19.086339 475 tablet_bootstrap.cc:492] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: No bootstrap required, opened a new log
I20260620 23:06:19.086408 475 ts_tablet_manager.cc:1397] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:06:19.087414 474 raft_consensus.cc:348] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.087553 474 raft_consensus.cc:374] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:19.087596 474 raft_consensus.cc:729] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7ab3cfb5564290ada93f93290bc40d, State: Initialized, Role: FOLLOWER
I20260620 23:06:19.087723 474 consensus_queue.cc:260] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.087692 475 raft_consensus.cc:348] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.087863 475 raft_consensus.cc:374] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:19.087908 475 raft_consensus.cc:729] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd4e21fe26f40fca224dec54e0004c8, State: Initialized, Role: FOLLOWER
I20260620 23:06:19.088049 474 ts_tablet_manager.cc:1428] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:06:19.088052 475 consensus_queue.cc:260] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.088207 332 heartbeater.cc:503] Master 127.17.188.126:45551 was elected leader, sending a full tablet report...
I20260620 23:06:19.088326 475 ts_tablet_manager.cc:1428] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:06:19.088336 464 heartbeater.cc:503] Master 127.17.188.126:45551 was elected leader, sending a full tablet report...
W20260620 23:06:19.183537 333 tablet.cc:2369] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:06:19.222489 478 raft_consensus.cc:482] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:19.222604 478 raft_consensus.cc:504] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.222903 478 leader_election.cc:288] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295), 378924fa0b92429fa1e0576446721836 (127.17.188.65:36745)
I20260620 23:06:19.226811 471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.223154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:52755 (local address 127.17.188.67:39295)
0620 23:06:19.223198 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:19.223200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:19.223303 (+ 103us) server_negotiation.cc:407] Connection header received
0620 23:06:19.223347 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:19.223348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:19.223357 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:19.223410 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:19.223597 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.224339 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.225883 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.226440 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.226531 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.226542 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:19.226704 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 23:06:19.226745 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:06:19.226831 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.223079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:39295 (local address 127.17.188.66:52755)
0620 23:06:19.223228 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.223237 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:19.223288 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:19.223410 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:19.223410 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:19.223412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:19.223509 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.223511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.224425 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.224425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.225789 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.225793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.226538 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.226539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.226602 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.226624 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 23:06:19.226698 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 23:06:19.226741 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":82,"thread_start_us":92,"threads_started":1}
I20260620 23:06:19.226919 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.223155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:47735 (local address 127.17.188.65:36745)
0620 23:06:19.223198 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:19.223200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:19.223209 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:06:19.223240 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:19.223240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:19.223265 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:19.223387 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:19.223604 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.224340 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.225883 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.226489 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.226549 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.226644 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:19.226790 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 23:06:19.226829 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:06:19.227092 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.223068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36745 (local address 127.17.188.66:47735)
0620 23:06:19.223134 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.223146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:19.223185 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:19.223410 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:19.223410 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:19.223412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:19.223509 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.223511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.224425 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.224425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.225789 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:19.225793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.226590 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.226591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:19.226698 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.226701 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:06:19.226781 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 23:06:19.226823 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":81}
I20260620 23:06:19.227239 418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88178fa48ab846e1a3b3d7eee25cfd8f" candidate_uuid: "fb7ab3cfb5564290ada93f93290bc40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd4e21fe26f40fca224dec54e0004c8" is_pre_election: true
I20260620 23:06:19.227360 32616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88178fa48ab846e1a3b3d7eee25cfd8f" candidate_uuid: "fb7ab3cfb5564290ada93f93290bc40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378924fa0b92429fa1e0576446721836" is_pre_election: true
I20260620 23:06:19.227413 418 raft_consensus.cc:2454] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb7ab3cfb5564290ada93f93290bc40d in term 0.
I20260620 23:06:19.227527 32616 raft_consensus.cc:2454] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb7ab3cfb5564290ada93f93290bc40d in term 0.
I20260620 23:06:19.227651 32688 leader_election.cc:302] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcd4e21fe26f40fca224dec54e0004c8, fb7ab3cfb5564290ada93f93290bc40d; no voters:
I20260620 23:06:19.227772 478 raft_consensus.cc:2781] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:06:19.227841 478 raft_consensus.cc:482] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:06:19.227864 478 raft_consensus.cc:3037] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:19.228549 478 raft_consensus.cc:504] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.228682 478 leader_election.cc:288] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [CANDIDATE]: Term 1 election: Requested vote from peers bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295), 378924fa0b92429fa1e0576446721836 (127.17.188.65:36745)
I20260620 23:06:19.228843 32616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88178fa48ab846e1a3b3d7eee25cfd8f" candidate_uuid: "fb7ab3cfb5564290ada93f93290bc40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378924fa0b92429fa1e0576446721836"
I20260620 23:06:19.228843 418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88178fa48ab846e1a3b3d7eee25cfd8f" candidate_uuid: "fb7ab3cfb5564290ada93f93290bc40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd4e21fe26f40fca224dec54e0004c8"
I20260620 23:06:19.228924 32616 raft_consensus.cc:3037] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:19.228922 418 raft_consensus.cc:3037] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:19.229789 32616 raft_consensus.cc:2454] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb7ab3cfb5564290ada93f93290bc40d in term 1.
I20260620 23:06:19.229789 418 raft_consensus.cc:2454] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb7ab3cfb5564290ada93f93290bc40d in term 1.
I20260620 23:06:19.230036 32688 leader_election.cc:302] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcd4e21fe26f40fca224dec54e0004c8, fb7ab3cfb5564290ada93f93290bc40d; no voters:
I20260620 23:06:19.230176 478 raft_consensus.cc:2781] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:19.230382 478 raft_consensus.cc:686] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 1 LEADER]: Becoming Leader. State: Replica: fb7ab3cfb5564290ada93f93290bc40d, State: Running, Role: LEADER
I20260620 23:06:19.230494 478 consensus_queue.cc:237] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.231364 32473 catalog_manager.cc:5495] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d reported cstate change: term changed from 0 to 1, leader changed from <none> to fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66). New cstate: current_term: 1 leader_uuid: "fb7ab3cfb5564290ada93f93290bc40d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } health_report { overall_health: UNKNOWN } } }
I20260620 23:06:19.259846 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.257657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.17.188.66:33871)
0620 23:06:19.257698 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:19.257700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:19.257709 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:06:19.257773 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:19.257774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:19.257795 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:19.257867 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:06:19.257979 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.258406 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.259129 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:19.259315 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:19.259389 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:19.259539 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:06:19.259631 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:06:19.259724 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:19.259743 (+ 19us) server_negotiation.cc:299] Negotiation successful
0620 23:06:19.259762 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:06:19.261376 418 raft_consensus.cc:1261] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [term 1 FOLLOWER]: Refusing update from remote peer fb7ab3cfb5564290ada93f93290bc40d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:19.261457 32616 raft_consensus.cc:1261] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Refusing update from remote peer fb7ab3cfb5564290ada93f93290bc40d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:19.261687 481 consensus_queue.cc:1035] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:19.261804 478 consensus_queue.cc:1035] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [LEADER]: Connected to new peer: Peer: permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:19.263880 484 mvcc.cc:205] Tried to move back new op lower bound from 7299058807852367872 to 7299058807728881664. Current Snapshot: MvccSnapshot[applied={T|T < 7299058807852367872}]
I20260620 23:06:19.264066 485 mvcc.cc:205] Tried to move back new op lower bound from 7299058807852367872 to 7299058807728881664. Current Snapshot: MvccSnapshot[applied={T|T < 7299058807852367872}]
I20260620 23:06:19.266394 32473 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:19.266479 32473 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:19.267122 32473 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Sending DeleteTablet for 3 replicas of tablet 88178fa48ab846e1a3b3d7eee25cfd8f
I20260620 23:06:19.267460 32596 tablet_service.cc:1514] Processing DeleteTablet for tablet 88178fa48ab846e1a3b3d7eee25cfd8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:19 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:49018
I20260620 23:06:19.267481 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 32512
I20260620 23:06:19.267464 32734 tablet_service.cc:1514] Processing DeleteTablet for tablet 88178fa48ab846e1a3b3d7eee25cfd8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:19 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:35668
I20260620 23:06:19.267541 32671 tablet_replica.cc:330] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: stopping tablet replica
I20260620 23:06:19.267587 32671 raft_consensus.cc:2229] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:06:19.267628 32671 raft_consensus.cc:2258] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:19.268306 494 tablet_replica.cc:330] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: stopping tablet replica
I20260620 23:06:19.268435 494 raft_consensus.cc:2229] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:06:19.268563 494 raft_consensus.cc:2258] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:19.268975 494 ts_tablet_manager.cc:1905] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:19.269547 398 tablet_service.cc:1514] Processing DeleteTablet for tablet 88178fa48ab846e1a3b3d7eee25cfd8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:19 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:33146
I20260620 23:06:19.270145 495 tablet_replica.cc:330] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: stopping tablet replica
I20260620 23:06:19.270265 495 raft_consensus.cc:2229] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:06:19.270406 495 raft_consensus.cc:2258] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:19.270656 494 ts_tablet_manager.cc:1918] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:19.270730 494 log.cc:1196] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/88178fa48ab846e1a3b3d7eee25cfd8f
I20260620 23:06:19.270979 495 ts_tablet_manager.cc:1905] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:19.271020 494 ts_tablet_manager.cc:1939] T 88178fa48ab846e1a3b3d7eee25cfd8f P fb7ab3cfb5564290ada93f93290bc40d: Deleting consensus metadata
I20260620 23:06:19.271359 32460 catalog_manager.cc:4841] TS fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871): tablet 88178fa48ab846e1a3b3d7eee25cfd8f (table test-table [id=6546496b82024090b65d4f0d0f1cc453]) successfully deleted
I20260620 23:06:19.272315 495 ts_tablet_manager.cc:1918] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:19.272408 495 log.cc:1196] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/88178fa48ab846e1a3b3d7eee25cfd8f
I20260620 23:06:19.272713 495 ts_tablet_manager.cc:1939] T 88178fa48ab846e1a3b3d7eee25cfd8f P bcd4e21fe26f40fca224dec54e0004c8: Deleting consensus metadata
W20260620 23:06:19.273162 32460 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:36745 (error 108)
I20260620 23:06:19.273201 32459 catalog_manager.cc:4841] TS bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295): tablet 88178fa48ab846e1a3b3d7eee25cfd8f (table test-table [id=6546496b82024090b65d4f0d0f1cc453]) successfully deleted
I20260620 23:06:19.273788 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.273657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36745 (local address 127.0.0.1:49024)
0620 23:06:19.273726 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.273761 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:36745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 23:06:19.273916 32460 catalog_manager.cc:4584] TS 378924fa0b92429fa1e0576446721836 (127.17.188.65:36745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88178fa48ab846e1a3b3d7eee25cfd8f: Network error: Client connection negotiation failed: client connection to 127.17.188.65:36745: connect: Connection refused (error 111)
I20260620 23:06:19.274293 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 32667
I20260620 23:06:19.279035 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 335
I20260620 23:06:19.283154 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 32441
I20260620 23:06:19.288141 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45551
--webserver_interface=127.17.188.126
--webserver_port=42067
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:45551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:19.367506 497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:19.367743 497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:19.367784 497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:19.368892 497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:19.368963 497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:19.368989 497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:19.369024 497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:19.369057 497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:19.370530 497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:45551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:19.370792 497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:19.371001 497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:19.375803 502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:19.375825 505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:19.375872 497 server_base.cc:1034] running on GCE node
W20260620 23:06:19.375859 503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:19.376400 497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:19.376686 497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:19.377854 497 hybrid_clock.cc:648] HybridClock initialized: now 1781996779377853 us; error 44 us; skew 500 ppm
Jun 20 23:06:19 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:19.380164 497 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:06:19.380554 497 webserver.cc:460] Webserver started at http://127.17.188.126:42067/ using document root <none> and password file <none>
I20260620 23:06:19.380790 497 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:19.380859 497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:19.382189 497 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:19.382829 512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.382998 497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:19.383095 497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "523c075900a947dc9016a4330d826d6e"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "d17e8788e00d2e477a0a5f8552a38124"
server_key_iv: "4ac0978911ef0673053c9fa561038e88"
server_key_version: "encryptionkey@0"
I20260620 23:06:19.383392 497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:19.402226 497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:19.403992 497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:19.404147 497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:19.407907 497 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:45551
I20260620 23:06:19.407941 564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:45551 every 8 connection(s)
I20260620 23:06:19.408301 497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:06:19.409875 565 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:06:19.410575 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Bootstrap starting.
I20260620 23:06:19.412134 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 497
I20260620 23:06:19.412487 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:36745
--local_ip_for_outbound_sockets=127.17.188.65
--tserver_master_addrs=127.17.188.126:45551
--webserver_port=41487
--webserver_interface=127.17.188.65
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:19.419107 565 log.cc:824] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:19.421165 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:06:19.421484 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e: Bootstrap complete.
I20260620 23:06:19.423616 565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:19.423839 565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 523c075900a947dc9016a4330d826d6e, State: Initialized, Role: FOLLOWER
I20260620 23:06:19.424021 565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:19.424094 565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:06:19.424144 565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:06:19.424188 565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 2 FOLLOWER]: Advancing to term 3
I20260620 23:06:19.425114 565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:19.425228 565 leader_election.cc:302] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 523c075900a947dc9016a4330d826d6e; no voters:
I20260620 23:06:19.425429 565 leader_election.cc:288] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 23:06:19.425463 569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 3 FOLLOWER]: Leader election won for term 3
I20260620 23:06:19.425562 569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [term 3 LEADER]: Becoming Leader. State: Replica: 523c075900a947dc9016a4330d826d6e, State: Running, Role: LEADER
I20260620 23:06:19.425643 569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } }
I20260620 23:06:19.425691 565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:19.425895 569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 523c075900a947dc9016a4330d826d6e. Latest consensus state: current_term: 3 leader_uuid: "523c075900a947dc9016a4330d826d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } } }
I20260620 23:06:19.426043 569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:19.425897 570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "523c075900a947dc9016a4330d826d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c075900a947dc9016a4330d826d6e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 45551 } } }
I20260620 23:06:19.426199 570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523c075900a947dc9016a4330d826d6e [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:19.426237 574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:06:19.426993 574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:06:19.427093 574 catalog_manager.cc:1259] Loaded cluster ID: 31c4c220998a4f40a7b859d743c4eb3e
I20260620 23:06:19.427126 574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 23:06:19.427327 574 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2c7f0e19ad google::LogMessage::Fail() at ??:0
@ 0x7f2c7f0e488a google::LogMessage::SendToLog() at ??:0
@ 0x7f2c7f0e14de google::LogMessage::Flush() at ??:0
@ 0x7f2c7f0e2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2c870020f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2c8700b8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2c80faf0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2c80fa4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2c81e086db start_thread at ??:0
@ 0x7f2c7dbdf61f clone at ??:0
W20260620 23:06:19.500469 567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:19.500674 567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:19.500711 567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:19.501892 567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:19.501950 567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:19.501994 567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:06:19.503371 567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:36745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=41487
--tserver_master_addrs=127.17.188.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:19.503615 567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:19.503823 567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:19.508621 601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:19.508667 602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:19.508730 567 server_base.cc:1034] running on GCE node
W20260620 23:06:19.508759 604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:19.509018 567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:19.509302 567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:19.510466 567 hybrid_clock.cc:648] HybridClock initialized: now 1781996779510454 us; error 32 us; skew 500 ppm
Jun 20 23:06:19 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:19.513087 567 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:06:19.513585 567 webserver.cc:460] Webserver started at http://127.17.188.65:41487/ using document root <none> and password file <none>
I20260620 23:06:19.513825 567 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:19.513911 567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:19.515220 567 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:19.515897 611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.516050 567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:19.516158 567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "378924fa0b92429fa1e0576446721836"
format_stamp: "Formatted at 2026-06-20 23:06:17 on dist-test-slave-zpss"
server_key: "c0b595daed0ff6d31b401d46b6a42dd6"
server_key_iv: "214545342f66ab82a62eacda668dfb61"
server_key_version: "encryptionkey@0"
I20260620 23:06:19.516503 567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:19.533164 567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:19.535243 567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:19.535379 567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:19.535611 567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:19.536067 618 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 23:06:19.537927 567 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 23:06:19.538010 567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 23:06:19.538058 567 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 23:06:19.538693 567 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 23:06:19.538748 567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 23:06:19.538805 618 tablet_bootstrap.cc:492] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Bootstrap starting.
I20260620 23:06:19.546612 567 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:36745
I20260620 23:06:19.546758 725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:36745 every 8 connection(s)
I20260620 23:06:19.547031 567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:06:19.548825 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 567
I20260620 23:06:19.549132 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:33871
--local_ip_for_outbound_sockets=127.17.188.66
--tserver_master_addrs=127.17.188.126:45551
--webserver_port=37211
--webserver_interface=127.17.188.66
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:19.550099 618 log.cc:824] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:19.553083 618 tablet_bootstrap.cc:492] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:06:19.553400 618 tablet_bootstrap.cc:492] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Bootstrap complete.
I20260620 23:06:19.554627 618 ts_tablet_manager.cc:1397] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260620 23:06:19.556025 618 raft_consensus.cc:348] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.556317 618 raft_consensus.cc:729] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378924fa0b92429fa1e0576446721836, State: Initialized, Role: FOLLOWER
I20260620 23:06:19.556452 618 consensus_queue.cc:260] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:19.556695 618 ts_tablet_manager.cc:1428] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260620 23:06:19.629459 729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:19.629673 729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:19.629709 729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:19.630878 729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:19.630941 729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:19.630987 729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:06:19.633919 729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:33871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=37211
--tserver_master_addrs=127.17.188.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:19.634171 729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:19.634425 729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:19.636366 728 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:06:19.547740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:52633)
0620 23:06:19.547920 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.547962 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:19.548433 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:19.636182 (+ 87749us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
W20260620 23:06:19.636607 726 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:45551 (0 consecutive failures): Network error: Failed to ping master at 127.17.188.126:45551: Client connection negotiation failed: client connection to 127.17.188.126:45551: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:06:19.636981 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.636852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:35195)
0620 23:06:19.636922 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.636960 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260620 23:06:19.637306 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.637215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:36675)
0620 23:06:19.637261 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.637286 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 23:06:19.637450 726 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:06:19.640174 737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:19.640164 739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:19.640148 736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:19.640478 729 server_base.cc:1034] running on GCE node
I20260620 23:06:19.640647 729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:19.640893 729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:19.642037 729 hybrid_clock.cc:648] HybridClock initialized: now 1781996779642022 us; error 28 us; skew 500 ppm
Jun 20 23:06:19 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:19.644371 729 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:06:19.644743 729 webserver.cc:460] Webserver started at http://127.17.188.66:37211/ using document root <none> and password file <none>
I20260620 23:06:19.644930 729 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:19.644971 729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:19.646255 729 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:19.646798 746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.646979 729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.647044 729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "fb7ab3cfb5564290ada93f93290bc40d"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "85cc28e43ca350eefe31cba61b771711"
server_key_iv: "e628661e2714710c30acf3cf565059f6"
server_key_version: "encryptionkey@0"
I20260620 23:06:19.647307 729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:19.686921 729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:19.689616 729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:19.689754 729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:19.689968 729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:19.690285 729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:19.690322 729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.690392 729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:19.690425 729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.697149 729 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:33871
I20260620 23:06:19.697198 859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:33871 every 8 connection(s)
I20260620 23:06:19.697664 729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:06:19.698881 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.698463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:57591)
0620 23:06:19.698625 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.698706 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
W20260620 23:06:19.699301 860 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:45551 (0 consecutive failures): Network error: Failed to ping master at 127.17.188.126:45551: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
I20260620 23:06:19.699643 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.699536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:52275)
0620 23:06:19.699580 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.699610 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 23:06:19.699926 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.699847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:42393)
0620 23:06:19.699887 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.699908 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 23:06:19.700026 860 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:06:19.704394 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 729
I20260620 23:06:19.704680 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:39295
--local_ip_for_outbound_sockets=127.17.188.67
--tserver_master_addrs=127.17.188.126:45551
--webserver_port=46509
--webserver_interface=127.17.188.67
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:19.781169 863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:19.781404 863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:19.781486 863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:19.782598 863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:19.782660 863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:19.782702 863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:06:19.784080 863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:39295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=46509
--tserver_master_addrs=127.17.188.126:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:19.784329 863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:19.784582 863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:19.789427 871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:19.789527 863 server_base.cc:1034] running on GCE node
W20260620 23:06:19.789453 868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:19.789485 869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:19.789811 863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:19.790048 863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:19.791186 863 hybrid_clock.cc:648] HybridClock initialized: now 1781996779791176 us; error 28 us; skew 500 ppm
Jun 20 23:06:19 dist-test-slave-zpss krb5kdc[31897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:19.793627 863 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:06:19.793969 863 webserver.cc:460] Webserver started at http://127.17.188.67:46509/ using document root <none> and password file <none>
I20260620 23:06:19.794162 863 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:19.794199 863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:19.795351 863 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:19.795871 878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.796039 863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.796128 863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "bcd4e21fe26f40fca224dec54e0004c8"
format_stamp: "Formatted at 2026-06-20 23:06:18 on dist-test-slave-zpss"
server_key: "282ba908e0d279a3efa7524def376fb0"
server_key_iv: "5508ffa891ee47853d88b50a2ec45e1d"
server_key_version: "encryptionkey@0"
I20260620 23:06:19.796418 863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:19.813483 863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:19.815742 863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:19.815908 863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:19.816112 863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:19.816383 863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:19.816432 863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.816470 863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:19.816497 863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:19.822256 863 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:39295
I20260620 23:06:19.822290 991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:39295 every 8 connection(s)
I20260620 23:06:19.822599 863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:06:19.823550 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.823230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:33179)
0620 23:06:19.823360 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.823427 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
W20260620 23:06:19.823943 992 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:45551 (0 consecutive failures): Network error: Failed to ping master at 127.17.188.126:45551: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
I20260620 23:06:19.824182 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.824102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:51851)
0620 23:06:19.824142 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.824167 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 23:06:19.824452 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:19.824370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:60647)
0620 23:06:19.824406 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:19.824427 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 23:06:19.824569 992 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:06:19.829278 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 863
I20260620 23:06:20.638055 996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:20.637832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:42033)
0620 23:06:20.637976 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:20.638009 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260620 23:06:20.700557 997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:20.700325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:49635)
0620 23:06:20.700460 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:20.700498 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260620 23:06:20.824994 998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:20.824827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:34465)
0620 23:06:20.824926 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:20.824953 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260620 23:06:20.963490 999 raft_consensus.cc:482] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:20.963605 999 raft_consensus.cc:504] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:20.963976 999 leader_election.cc:288] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295), fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871)
Jun 20 23:06:20 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.66@KRBTEST.COM
Jun 20 23:06:20 dist-test-slave-zpss krb5kdc[31897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.67@KRBTEST.COM
I20260620 23:06:20.978906 996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:20.964112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:33871 (local address 127.17.188.65:35611)
0620 23:06:20.964167 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:20.964177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:20.964227 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:20.965270 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:20.965273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:20.965831 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:20.966087 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:20.966094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:20.967651 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:20.967652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:20.968940 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:20.968943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:20.969761 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:20.969762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:20.969885 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:20.973714 (+ 3829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:20.973722 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:20.975325 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:20.977241 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:20.977247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:20.977249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:20.977548 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:20.977777 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:20.977777 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:20.977778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:20.977919 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:20.978373 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:20.978378 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:20.978523 (+ 145us) client_negotiation.cc:769] Sending connection context
0620 23:06:20.978637 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 23:06:20.978779 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":56}
I20260620 23:06:20.978941 1001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:20.964224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:35611 (local address 127.17.188.66:33871)
0620 23:06:20.964339 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:20.964352 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:20.964397 (+ 45us) server_negotiation.cc:407] Connection header received
0620 23:06:20.964993 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:20.964996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:20.965055 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:20.965204 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:20.966208 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:20.967548 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:20.969027 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:20.969657 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:20.969895 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:20.975440 (+ 5545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:20.975443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:20.975444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:20.975453 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:20.977157 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:20.977627 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:20.977630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:20.977630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:20.977711 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:20.977993 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:20.977994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:20.977994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:20.978279 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:20.978374 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:20.978696 (+ 322us) server_negotiation.cc:299] Negotiation successful
0620 23:06:20.978821 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
I20260620 23:06:20.979089 1000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:20.964154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:39295 (local address 127.17.188.65:52055)
0620 23:06:20.964253 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:20.964264 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:20.964316 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:20.965270 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:20.965273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:20.965831 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:20.966087 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:20.966094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:20.967659 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:20.967660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:20.968940 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:20.968943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:20.969879 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:20.969881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:20.969943 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:20.973714 (+ 3771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:20.973723 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:20.975835 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:20.977903 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:20.977904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:20.977904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:20.978113 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:20.978322 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:20.978323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:20.978323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:20.978390 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:20.978802 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:20.978803 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:20.978843 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 23:06:20.978911 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:06:20.978998 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":81,"thread_start_us":42,"threads_started":1}
I20260620 23:06:20.979233 1002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:20.964233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:52055 (local address 127.17.188.67:39295)
0620 23:06:20.964405 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:20.964430 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:20.964456 (+ 26us) server_negotiation.cc:407] Connection header received
0620 23:06:20.964993 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:20.964996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:20.965055 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:20.965204 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:20.966208 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:20.967548 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:20.969029 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:20.969674 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:20.969788 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:20.975928 (+ 6140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:20.975932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:20.975933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:20.975943 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:20.977832 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:20.978185 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:20.978187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:20.978187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:20.978258 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:20.978450 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:20.978450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:20.978451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:20.978724 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:20.978797 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:20.978989 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 23:06:20.979131 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
W20260620 23:06:20.979907 614 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
W20260620 23:06:20.979983 613 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
I20260620 23:06:20.980080 613 leader_election.cc:302] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 378924fa0b92429fa1e0576446721836; no voters: bcd4e21fe26f40fca224dec54e0004c8, fb7ab3cfb5564290ada93f93290bc40d
I20260620 23:06:20.980196 999 raft_consensus.cc:2726] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:06:21.638629 1003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:21.638445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:53295)
0620 23:06:21.638555 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:21.638585 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260620 23:06:21.701138 1004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:21.700959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:49647)
0620 23:06:21.701075 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:21.701105 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":54,"threads_started":1}
I20260620 23:06:21.825670 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:21.825449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:37807)
0620 23:06:21.825596 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:21.825624 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260620 23:06:22.639365 1006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:22.639124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:40267)
0620 23:06:22.639250 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:22.639278 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260620 23:06:22.701736 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:22.701509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:58161)
0620 23:06:22.701658 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:22.701693 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260620 23:06:22.826339 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:22.826152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:47659)
0620 23:06:22.826273 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:22.826303 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260620 23:06:23.189182 1009 raft_consensus.cc:482] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:23.189299 1009 raft_consensus.cc:504] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:23.189533 1009 leader_election.cc:288] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295), fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871)
W20260620 23:06:23.189934 613 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
W20260620 23:06:23.189997 614 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
I20260620 23:06:23.190028 614 leader_election.cc:302] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 378924fa0b92429fa1e0576446721836; no voters: bcd4e21fe26f40fca224dec54e0004c8, fb7ab3cfb5564290ada93f93290bc40d
I20260620 23:06:23.190135 1009 raft_consensus.cc:2726] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:06:23.640014 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:23.639809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:58465)
0620 23:06:23.639935 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:23.639970 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260620 23:06:23.702332 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:23.702157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:49003)
0620 23:06:23.702263 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:23.702299 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260620 23:06:23.827020 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:23.826851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:55709)
0620 23:06:23.826957 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:23.826987 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260620 23:06:24.640618 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:24.640406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:46429)
0620 23:06:24.640530 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:24.640574 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":55,"threads_started":1}
I20260620 23:06:24.702867 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:24.702703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:41961)
0620 23:06:24.702807 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:24.702836 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":51,"threads_started":1}
I20260620 23:06:24.827600 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:24.827404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:53065)
0620 23:06:24.827513 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:24.827563 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260620 23:06:24.846439 1016 raft_consensus.cc:482] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:24.846563 1016 raft_consensus.cc:504] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:24.846761 1016 leader_election.cc:288] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295), fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871)
W20260620 23:06:24.847124 613 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
W20260620 23:06:24.847178 614 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
I20260620 23:06:24.847211 614 leader_election.cc:302] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 378924fa0b92429fa1e0576446721836; no voters: bcd4e21fe26f40fca224dec54e0004c8, fb7ab3cfb5564290ada93f93290bc40d
I20260620 23:06:24.847295 1016 raft_consensus.cc:2726] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:06:25.641183 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:25.640991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:55353)
0620 23:06:25.641112 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:25.641139 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260620 23:06:25.703409 1018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:25.703237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:36697)
0620 23:06:25.703347 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:25.703377 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":52,"threads_started":1}
I20260620 23:06:25.828348 1019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:25.828176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:39513)
0620 23:06:25.828286 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:25.828313 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260620 23:06:26.641789 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:26.641594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:53973)
0620 23:06:26.641710 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:26.641742 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260620 23:06:26.703985 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:26.703800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:55547)
0620 23:06:26.703911 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:26.703951 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":53,"threads_started":1}
I20260620 23:06:26.828877 1022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:26.828706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:32877)
0620 23:06:26.828814 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:26.828842 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":52,"threads_started":1}
I20260620 23:06:27.642378 1023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:27.642186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:56577)
0620 23:06:27.642298 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:27.642333 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
I20260620 23:06:27.704638 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:27.704435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:56291)
0620 23:06:27.704563 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:27.704597 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260620 23:06:27.829512 1025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:27.829283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:40651)
0620 23:06:27.829439 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:27.829473 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260620 23:06:28.359418 1026 raft_consensus.cc:482] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:28.359525 1026 raft_consensus.cc:504] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:28.359736 1026 leader_election.cc:288] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295), fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871)
W20260620 23:06:28.360215 613 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
W20260620 23:06:28.360291 614 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
I20260620 23:06:28.360386 614 leader_election.cc:302] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 378924fa0b92429fa1e0576446721836; no voters: bcd4e21fe26f40fca224dec54e0004c8, fb7ab3cfb5564290ada93f93290bc40d
I20260620 23:06:28.360468 1026 raft_consensus.cc:2726] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:06:28.642989 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:28.642769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:53183)
0620 23:06:28.642886 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:28.642940 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260620 23:06:28.705298 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:28.705100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:49521)
0620 23:06:28.705214 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:28.705249 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260620 23:06:28.830224 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:28.830019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:45633)
0620 23:06:28.830157 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:28.830189 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260620 23:06:29.643649 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:29.643461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:39709)
0620 23:06:29.643571 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:29.643604 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260620 23:06:29.705931 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:29.705738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:37139)
0620 23:06:29.705866 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:29.705897 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260620 23:06:29.830905 1032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:29.830615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:46133)
0620 23:06:29.830839 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:29.830872 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":66,"threads_started":1}
I20260620 23:06:30.644317 1033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:30.644112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:47151)
0620 23:06:30.644240 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:30.644274 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260620 23:06:30.706542 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:30.706348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:33067)
0620 23:06:30.706475 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:30.706506 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260620 23:06:30.831682 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:30.831408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:33677)
0620 23:06:30.831613 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:30.831646 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":53,"threads_started":1}
I20260620 23:06:30.899706 1036 raft_consensus.cc:482] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:30.899812 1036 raft_consensus.cc:504] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e21fe26f40fca224dec54e0004c8" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 39295 } } peers { permanent_uuid: "fb7ab3cfb5564290ada93f93290bc40d" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 33871 } } peers { permanent_uuid: "378924fa0b92429fa1e0576446721836" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36745 } }
I20260620 23:06:30.900005 1036 leader_election.cc:288] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295), fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871)
W20260620 23:06:30.900342 613 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e21fe26f40fca224dec54e0004c8 (127.17.188.67:39295): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
W20260620 23:06:30.900398 614 leader_election.cc:341] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb7ab3cfb5564290ada93f93290bc40d (127.17.188.66:33871): Not found: Tablet not found: 88178fa48ab846e1a3b3d7eee25cfd8f
I20260620 23:06:30.900429 614 leader_election.cc:302] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 378924fa0b92429fa1e0576446721836; no voters: bcd4e21fe26f40fca224dec54e0004c8, fb7ab3cfb5564290ada93f93290bc40d
I20260620 23:06:30.900506 1036 raft_consensus.cc:2726] T 88178fa48ab846e1a3b3d7eee25cfd8f P 378924fa0b92429fa1e0576446721836 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:06:31.644939 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:31.644731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:40677)
0620 23:06:31.644860 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:31.644892 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260620 23:06:31.707105 1038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:31.706934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:51675)
0620 23:06:31.707039 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:31.707069 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260620 23:06:31.832288 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:31.832110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:41201)
0620 23:06:31.832217 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:31.832254 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260620 23:06:32.645622 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:32.645399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:41771)
0620 23:06:32.645541 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:32.645576 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260620 23:06:32.707679 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:32.707501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:48959)
0620 23:06:32.707614 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:32.707642 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260620 23:06:32.832940 1042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:32.832728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:59613)
0620 23:06:32.832863 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:32.832898 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260620 23:06:33.646320 1043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:33.646038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:39849)
0620 23:06:33.646169 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:33.646246 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260620 23:06:33.708271 1044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:33.708065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:47297)
0620 23:06:33.708200 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:33.708230 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260620 23:06:33.833600 1045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:33.833409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:57781)
0620 23:06:33.833535 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:33.833566 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260620 23:06:34.647327 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:34.647057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.65:54655)
0620 23:06:34.647227 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:34.647272 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":119,"threads_started":1}
I20260620 23:06:34.708891 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:34.708703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.66:57453)
0620 23:06:34.708829 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:34.708857 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260620 23:06:34.834318 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:34.834111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:45551 (local address 127.17.188.67:59039)
0620 23:06:34.834240 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:34.834276 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:45551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260620 23:06:34.834738 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 567
I20260620 23:06:34.839527 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 729
I20260620 23:06:34.843802 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 863
2026-06-20T23:06:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18383 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:06:34 dist-test-slave-zpss krb5kdc[1051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:06:34 dist-test-slave-zpss krb5kdc[1051](info): set up 2 sockets
Jun 20 23:06:34 dist-test-slave-zpss krb5kdc[1051](info): commencing operation
krb5kdc: starting...
W20260620 23:06:35.869206 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:06:35 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:06:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:06:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:06:35.975303 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:40003
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:40003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:36.049515 1064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:36.049737 1064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:36.049772 1064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:36.050817 1064 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 23:06:36.050882 1064 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 23:06:36.050901 1064 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 23:06:36.050917 1064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:36.050943 1064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:36.050971 1064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:36.050992 1064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:36.051038 1064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:36.052426 1064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:40003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:40003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:36.052687 1064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:36.052893 1064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:36.057447 1069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:36.057449 1070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:36.057425 1072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:36.057674 1064 server_base.cc:1034] running on GCE node
I20260620 23:06:36.057906 1064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:36.058189 1064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:36.059378 1064 hybrid_clock.cc:648] HybridClock initialized: now 1781996796059366 us; error 43 us; skew 500 ppm
Jun 20 23:06:36 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:36.061563 1064 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:06:36.061985 1064 webserver.cc:460] Webserver started at http://127.17.188.126:38341/ using document root <none> and password file <none>
I20260620 23:06:36.062191 1064 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:36.062251 1064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:36.062386 1064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:36.063418 1064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "3d56aeed930749609d2d2df5fb33f1fa"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "82132cf4cb46fb00eac75c1f73fa65f7"
server_key_iv: "3427823e9f4ff023ab6dc152d139d5f5"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.063781 1064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "3d56aeed930749609d2d2df5fb33f1fa"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "82132cf4cb46fb00eac75c1f73fa65f7"
server_key_iv: "3427823e9f4ff023ab6dc152d139d5f5"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.065059 1064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:06:36.065816 1079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.065982 1064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.066078 1064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "3d56aeed930749609d2d2df5fb33f1fa"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "82132cf4cb46fb00eac75c1f73fa65f7"
server_key_iv: "3427823e9f4ff023ab6dc152d139d5f5"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.066161 1064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:36.080255 1064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:36.081877 1064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:36.082077 1064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:36.085930 1064 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:40003
I20260620 23:06:36.085958 1131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:40003 every 8 connection(s)
I20260620 23:06:36.086265 1064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:06:36.087013 1132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:36.088919 1132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Bootstrap starting.
I20260620 23:06:36.089259 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 1064
I20260620 23:06:36.089385 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:06:36.089505 1132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:36.089594 18161 external_mini_cluster.cc:1428] Setting key a83906dee16cd12ac0ed763559d04fdd
I20260620 23:06:36.089807 1132 log.cc:824] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:36.090479 1132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: No bootstrap required, opened a new log
I20260620 23:06:36.091984 1132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:06:36.092110 1132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:36.092165 1132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d56aeed930749609d2d2df5fb33f1fa, State: Initialized, Role: FOLLOWER
I20260620 23:06:36.092290 1132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:06:36.092392 1132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:06:36.092432 1132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:06:36.092483 1132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:36.093578 1132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:06:36.093698 1132 leader_election.cc:302] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d56aeed930749609d2d2df5fb33f1fa; no voters:
I20260620 23:06:36.093890 1132 leader_election.cc:288] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:06:36.093930 1137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:36.094115 1137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 1 LEADER]: Becoming Leader. State: Replica: 3d56aeed930749609d2d2df5fb33f1fa, State: Running, Role: LEADER
I20260620 23:06:36.094172 1132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:36.094218 1137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:06:36.094511 1137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d56aeed930749609d2d2df5fb33f1fa. Latest consensus state: current_term: 1 leader_uuid: "3d56aeed930749609d2d2df5fb33f1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } } }
I20260620 23:06:36.094579 1137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:36.094522 1138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d56aeed930749609d2d2df5fb33f1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } } }
I20260620 23:06:36.094746 1138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: This master's current role is: LEADER
W20260620 23:06:36.095983 1153 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:06:36.096036 1153 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:06:36.096077 1142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:06:36.096217 1142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:06:36 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:36.098675 1142 catalog_manager.cc:1347] Generated new cluster ID: 6cae1c575858489c93d7afb64519bfbd
I20260620 23:06:36.098724 1142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:06:36.102521 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.090046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43558 (local address 127.17.188.126:40003)
0620 23:06:36.090201 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.090209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.092322 (+ 2113us) server_negotiation.cc:407] Connection header received
0620 23:06:36.092487 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.092494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.092571 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.092695 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:36.093488 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.094800 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.095785 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.096263 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.096386 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.098996 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:36.099000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:36.099000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:36.099009 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:36.100874 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.101289 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.101291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.101292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.101428 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.101671 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.101671 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.101672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.101965 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:36.102095 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.102232 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.102349 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260620 23:06:36.114197 1142 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:06:36.114638 1142 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:06:36.122305 1142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Generated new TSK 0
I20260620 23:06:36.122462 1142 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:06:36.147732 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:40003
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:36.226228 1158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:36.226406 1158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:36.226428 1158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:36.227456 1158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:36.227487 1158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:36.227511 1158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:06:36.228797 1158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:36.228978 1158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:36.229132 1158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:36.233888 1163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:36.233898 1166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:36.233956 1158 server_base.cc:1034] running on GCE node
W20260620 23:06:36.233898 1164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:36.234282 1158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:36.234516 1158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:36.235654 1158 hybrid_clock.cc:648] HybridClock initialized: now 1781996796235644 us; error 27 us; skew 500 ppm
Jun 20 23:06:36 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:36.237914 1158 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:06:36.238354 1158 webserver.cc:460] Webserver started at http://127.17.188.65:39037/ using document root <none> and password file <none>
I20260620 23:06:36.238554 1158 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:36.238595 1158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:36.238751 1158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:36.239672 1158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "b6b30fa096d74678a7e732e09451a96b"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "09dec22912f5d7578d8d11c2c8f4a745"
server_key_iv: "60bb287d6c170806a333105cabb9bc22"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.240007 1158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "b6b30fa096d74678a7e732e09451a96b"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "09dec22912f5d7578d8d11c2c8f4a745"
server_key_iv: "60bb287d6c170806a333105cabb9bc22"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.241235 1158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:06:36.241966 1173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.242138 1158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:36.242202 1158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "b6b30fa096d74678a7e732e09451a96b"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "09dec22912f5d7578d8d11c2c8f4a745"
server_key_iv: "60bb287d6c170806a333105cabb9bc22"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.242286 1158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:36.251380 1158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:36.252971 1158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:36.253140 1158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:36.253378 1158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:36.253685 1158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:36.253716 1158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.253736 1158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:36.253779 1158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.259527 1158 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:39691
I20260620 23:06:36.259593 1286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:39691 every 8 connection(s)
I20260620 23:06:36.259857 1158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:06:36.262068 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 1158
I20260620 23:06:36.262212 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:06:36.262455 18161 external_mini_cluster.cc:1428] Setting key 23f4e80338dffd7da7a73be8e2de8d6f
Jun 20 23:06:36 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:36.270431 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.260467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:59085)
0620 23:06:36.260617 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:36.260654 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:36.261060 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:36.261244 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:36.261247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:36.262130 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:36.262426 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.262433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.263406 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.263407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.264874 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.264877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.265362 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.265363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.265507 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.266011 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:36.266025 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:36.267396 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:36.269074 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:36.269082 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:36.269084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:36.269300 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:36.269563 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:36.269564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:36.269564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:36.269672 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:36.269971 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:36.269976 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:36.270078 (+ 102us) client_negotiation.cc:769] Sending connection context
0620 23:06:36.270178 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 23:06:36.270288 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260620 23:06:36.270459 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.260509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:59085 (local address 127.17.188.126:40003)
0620 23:06:36.260554 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.260556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.260732 (+ 176us) server_negotiation.cc:407] Connection header received
0620 23:06:36.261133 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.261134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.261146 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.261197 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:36.262572 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.263309 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.265003 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.265241 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.265355 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.267499 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:36.267501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:36.267501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:36.267511 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:36.268968 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.269419 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.269420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.269421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.269481 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.269754 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.269756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.269756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.269907 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:36.269976 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.270259 (+ 283us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.270346 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
I20260620 23:06:36.270917 1287 heartbeater.cc:347] Connected to a master server at 127.17.188.126:40003
I20260620 23:06:36.271015 1287 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:36.271175 1287 heartbeater.cc:511] Master 127.17.188.126:40003 requested a full tablet report, sending...
I20260620 23:06:36.271603 1096 ts_manager.cc:194] Registered new tserver with Master: b6b30fa096d74678a7e732e09451a96b (127.17.188.65:39691)
I20260620 23:06:36.272346 1096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:59085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:06:36.286139 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:40003
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:36.363801 1292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:36.364013 1292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:36.364053 1292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:36.365180 1292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:36.365247 1292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:36.365293 1292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:06:36.366739 1292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:36.366979 1292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:36.367198 1292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:36.372001 1298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:36.372169 1297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:36.372288 1292 server_base.cc:1034] running on GCE node
W20260620 23:06:36.372391 1300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:36.372613 1292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:36.372874 1292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:36.374006 1292 hybrid_clock.cc:648] HybridClock initialized: now 1781996796373995 us; error 27 us; skew 500 ppm
Jun 20 23:06:36 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:36.376283 1292 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:06:36.376706 1292 webserver.cc:460] Webserver started at http://127.17.188.66:36491/ using document root <none> and password file <none>
I20260620 23:06:36.376868 1292 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:36.376905 1292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:36.376984 1292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:36.377985 1292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "9967ffca7fb3469ebd123c88ef441cde"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "8fbf4aadff7ea7a9eaa3397d2e16f19d"
server_key_iv: "778fe534dc013b1f0af872e309435468"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.378274 1292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "9967ffca7fb3469ebd123c88ef441cde"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "8fbf4aadff7ea7a9eaa3397d2e16f19d"
server_key_iv: "778fe534dc013b1f0af872e309435468"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.379468 1292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:06:36.380093 1307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.380255 1292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:36.380340 1292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "9967ffca7fb3469ebd123c88ef441cde"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "8fbf4aadff7ea7a9eaa3397d2e16f19d"
server_key_iv: "778fe534dc013b1f0af872e309435468"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.380409 1292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:36.398147 1292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:36.399863 1292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:36.400010 1292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:36.400223 1292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:36.400498 1292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:36.400530 1292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.400550 1292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:36.400589 1292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.406203 1292 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:34783
I20260620 23:06:36.406239 1420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:34783 every 8 connection(s)
I20260620 23:06:36.406533 1292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:06:36.409830 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 1292
I20260620 23:06:36.409942 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:06:36.410143 18161 external_mini_cluster.cc:1428] Setting key a5956087d5548d83c0891357043cdbb7
Jun 20 23:06:36 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:36.415771 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.407241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:38013 (local address 127.17.188.126:40003)
0620 23:06:36.407327 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.407330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.407497 (+ 167us) server_negotiation.cc:407] Connection header received
0620 23:06:36.407995 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.407997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.408008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.408055 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:36.408957 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.409525 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.410687 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.410930 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.410999 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.413016 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:36.413017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:36.413018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:36.413023 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:36.414373 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.414832 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.414833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.414834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.414879 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.415160 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.415160 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.415161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.415251 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:36.415319 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.415648 (+ 329us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.415711 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 23:06:36.416085 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.407169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:38013)
0620 23:06:36.407372 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:36.407418 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:36.407890 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:36.408135 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:36.408139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:36.408680 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:36.408858 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.408862 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.409614 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.409616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.410588 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.410590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.411007 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.411008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.411112 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.411616 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:36.411631 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:36.412938 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:36.414456 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:36.414464 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:36.414466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:36.414754 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:36.414939 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:36.414940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:36.414940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:36.415090 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:36.415337 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:36.415343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:36.415508 (+ 165us) client_negotiation.cc:769] Sending connection context
0620 23:06:36.415630 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 23:06:36.415808 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260620 23:06:36.416594 1421 heartbeater.cc:347] Connected to a master server at 127.17.188.126:40003
I20260620 23:06:36.416692 1421 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:36.416853 1421 heartbeater.cc:511] Master 127.17.188.126:40003 requested a full tablet report, sending...
I20260620 23:06:36.417199 1096 ts_manager.cc:194] Registered new tserver with Master: 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783)
I20260620 23:06:36.417570 1096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:38013
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:06:36.433672 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:40003
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:36.511861 1426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:36.512048 1426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:36.512070 1426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:36.513198 1426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:36.513240 1426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:36.513290 1426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:06:36.514761 1426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:36.514993 1426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:36.515213 1426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:36.520196 1434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:36.520197 1432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:36.520306 1431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:36.520481 1426 server_base.cc:1034] running on GCE node
I20260620 23:06:36.520716 1426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:36.520972 1426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:36.522174 1426 hybrid_clock.cc:648] HybridClock initialized: now 1781996796522160 us; error 25 us; skew 500 ppm
Jun 20 23:06:36 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:36.524559 1426 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:06:36.525005 1426 webserver.cc:460] Webserver started at http://127.17.188.67:33701/ using document root <none> and password file <none>
I20260620 23:06:36.525238 1426 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:36.525316 1426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:36.525485 1426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:06:36.526454 1426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "16c5fa5d8bf04c4e95fd886479ceb85a"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "a0d5e0fee8ba2c1e6db7f5da154c4aab"
server_key_iv: "2bff0c579add74c2d914b7b9483afeec"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.526790 1426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "16c5fa5d8bf04c4e95fd886479ceb85a"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "a0d5e0fee8ba2c1e6db7f5da154c4aab"
server_key_iv: "2bff0c579add74c2d914b7b9483afeec"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.528015 1426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:06:36.528676 1441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.528838 1426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:06:36.528931 1426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "16c5fa5d8bf04c4e95fd886479ceb85a"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "a0d5e0fee8ba2c1e6db7f5da154c4aab"
server_key_iv: "2bff0c579add74c2d914b7b9483afeec"
server_key_version: "encryptionkey@0"
I20260620 23:06:36.529032 1426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:36.538621 1426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:36.540771 1426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:36.540948 1426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:36.541169 1426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:36.541497 1426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:36.541548 1426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.541580 1426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:36.541606 1426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:36.547376 1426 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:33015
I20260620 23:06:36.547456 1554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:33015 every 8 connection(s)
I20260620 23:06:36.547739 1426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:06:36.557674 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 1426
I20260620 23:06:36.557775 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:06:36.557998 18161 external_mini_cluster.cc:1428] Setting key 8affcad4c2900634479ddff03f666081
Jun 20 23:06:36 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:36.565567 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.557016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:52005 (local address 127.17.188.126:40003)
0620 23:06:36.557110 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.557112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.557123 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:06:36.557161 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.557162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.557172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.557238 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:36.558283 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.559040 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.560003 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.560205 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.560271 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.562596 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:36.562597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:36.562598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:36.562603 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:36.563966 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.564477 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.564478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.564478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.564562 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.564910 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.564911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.564911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.565035 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:36.565087 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.565439 (+ 352us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.565497 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 23:06:36.565853 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.549926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:52005)
0620 23:06:36.550129 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:36.550186 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:36.550822 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:36.557347 (+ 6525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:36.557352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:36.557952 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:36.558174 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.558181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.559126 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.559127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.559904 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.559906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.560276 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.560277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.560393 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.560861 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:36.560868 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:36.562514 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:36.564041 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:36.564048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:36.564050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:36.564378 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:36.564636 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:36.564637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:36.564637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:36.564826 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:36.565104 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:36.565109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:36.565284 (+ 175us) client_negotiation.cc:769] Sending connection context
0620 23:06:36.565419 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 23:06:36.565584 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":114,"threads_started":1}
I20260620 23:06:36.566334 1555 heartbeater.cc:347] Connected to a master server at 127.17.188.126:40003
I20260620 23:06:36.566438 1555 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:36.566620 1555 heartbeater.cc:511] Master 127.17.188.126:40003 requested a full tablet report, sending...
I20260620 23:06:36.566944 1096 ts_manager.cc:194] Registered new tserver with Master: 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015)
I20260620 23:06:36.567308 1096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:52005
I20260620 23:06:36.569705 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:06:36.577414 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.570986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.17.188.126:40003)
0620 23:06:36.571019 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.571021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.571162 (+ 141us) server_negotiation.cc:407] Connection header received
0620 23:06:36.571235 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.571236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.571245 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.571299 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:36.571854 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.572383 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.573313 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.573551 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.573622 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.574793 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:36.574794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:36.574794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:36.574801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:36.576201 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.576567 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.576568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.576568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.576620 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:36.576825 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:36.576826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:36.576826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:36.577008 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:36.577123 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.577227 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.577299 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:06:36.579102 1096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:06:36.580641 1096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:06:36.587208 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.582895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33015 (local address 127.0.0.1:55746)
0620 23:06:36.583001 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:36.583040 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:36.583099 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:36.583416 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:36.583419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:36.583420 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:36.583478 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.583480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.584764 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.584765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.586084 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.586086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.586944 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.586945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.587019 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.587032 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 23:06:36.587090 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 23:06:36.587136 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":39,"thread_start_us":59,"threads_started":1}
I20260620 23:06:36.587685 1566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.583051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.17.188.67:33015)
0620 23:06:36.583216 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.583221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.583231 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:06:36.583306 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.583309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.583323 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.583401 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:36.583544 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.584661 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.586172 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.586851 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.587474 (+ 623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.587533 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.587557 (+ 24us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.587598 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260620 23:06:36.587849 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.583107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:39691 (local address 127.0.0.1:33348)
0620 23:06:36.583245 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:36.583255 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:36.583298 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:36.583709 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:36.583710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:36.583713 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:36.583758 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.583760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.585017 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.585018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.586352 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.586355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.587601 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.587603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.587681 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.587687 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:06:36.587719 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 23:06:36.587762 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":96,"threads_started":1}
I20260620 23:06:36.588023 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.583022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39842 (local address 127.17.188.66:34783)
0620 23:06:36.583790 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.583802 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.583811 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:06:36.583891 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.583893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.583908 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.584001 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:36.584104 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.584909 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.586519 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.587220 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.587282 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.587345 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.587908 (+ 563us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.587947 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":89,"threads_started":1}
I20260620 23:06:36.588194 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.582901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:34783 (local address 127.0.0.1:39842)
0620 23:06:36.583190 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:36.583201 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:36.583255 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:36.583999 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:36.584000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:36.584001 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:36.584039 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.584049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.585016 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.585017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.586437 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.586439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.587381 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.587382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.587444 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.587448 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:06:36.587494 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 23:06:36.587528 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":65,"thread_start_us":53,"threads_started":1}
I20260620 23:06:36.588441 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.583345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33348 (local address 127.17.188.65:39691)
0620 23:06:36.583504 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.583508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.583518 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:06:36.583556 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.583558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.583574 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.583883 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:36.583953 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.584917 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.586458 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.587493 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.588231 (+ 738us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.588296 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.588323 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.588363 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260620 23:06:36.588372 1489 tablet_service.cc:1467] Processing CreateTablet for tablet d54c3990805e4561b4d30fb8b253f42a (DEFAULT_TABLE table=test-table [id=2266a2a528da466cbec2a043d9391977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:36.588711 1489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d54c3990805e4561b4d30fb8b253f42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:36.588858 1355 tablet_service.cc:1467] Processing CreateTablet for tablet d54c3990805e4561b4d30fb8b253f42a (DEFAULT_TABLE table=test-table [id=2266a2a528da466cbec2a043d9391977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:36.589033 1221 tablet_service.cc:1467] Processing CreateTablet for tablet d54c3990805e4561b4d30fb8b253f42a (DEFAULT_TABLE table=test-table [id=2266a2a528da466cbec2a043d9391977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:36.589200 1355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d54c3990805e4561b4d30fb8b253f42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:36.589306 1221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d54c3990805e4561b4d30fb8b253f42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:36.591288 1569 tablet_bootstrap.cc:492] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: Bootstrap starting.
I20260620 23:06:36.591965 1569 tablet_bootstrap.cc:654] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:36.592389 1571 tablet_bootstrap.cc:492] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Bootstrap starting.
I20260620 23:06:36.592885 1569 log.cc:824] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:36.593076 1571 tablet_bootstrap.cc:654] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:36.593676 1571 log.cc:824] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:36.593716 1569 tablet_bootstrap.cc:492] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: No bootstrap required, opened a new log
I20260620 23:06:36.593778 1569 ts_tablet_manager.cc:1397] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:06:36.594424 1571 tablet_bootstrap.cc:492] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: No bootstrap required, opened a new log
I20260620 23:06:36.594501 1571 ts_tablet_manager.cc:1397] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 23:06:36.595741 1569 raft_consensus.cc:348] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:36.595969 1569 raft_consensus.cc:374] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:36.596050 1569 raft_consensus.cc:729] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c5fa5d8bf04c4e95fd886479ceb85a, State: Initialized, Role: FOLLOWER
I20260620 23:06:36.596199 1569 consensus_queue.cc:260] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:36.596478 1555 heartbeater.cc:503] Master 127.17.188.126:40003 was elected leader, sending a full tablet report...
I20260620 23:06:36.596483 1569 ts_tablet_manager.cc:1428] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:06:36.596908 1571 raft_consensus.cc:348] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:36.597093 1571 raft_consensus.cc:374] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:36.597136 1571 raft_consensus.cc:729] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b30fa096d74678a7e732e09451a96b, State: Initialized, Role: FOLLOWER
I20260620 23:06:36.597258 1571 consensus_queue.cc:260] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:36.597573 1571 ts_tablet_manager.cc:1428] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:06:36.597668 1570 tablet_bootstrap.cc:492] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: Bootstrap starting.
I20260620 23:06:36.597719 1287 heartbeater.cc:503] Master 127.17.188.126:40003 was elected leader, sending a full tablet report...
I20260620 23:06:36.598330 1570 tablet_bootstrap.cc:654] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:36.598852 1570 log.cc:824] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:36.599599 1570 tablet_bootstrap.cc:492] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: No bootstrap required, opened a new log
I20260620 23:06:36.599665 1570 ts_tablet_manager.cc:1397] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:06:36.601760 1570 raft_consensus.cc:348] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:36.601912 1570 raft_consensus.cc:374] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:36.601939 1570 raft_consensus.cc:729] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9967ffca7fb3469ebd123c88ef441cde, State: Initialized, Role: FOLLOWER
I20260620 23:06:36.602025 1570 consensus_queue.cc:260] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:36.602241 1570 ts_tablet_manager.cc:1428] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:06:36.605289 1421 heartbeater.cc:503] Master 127.17.188.126:40003 was elected leader, sending a full tablet report...
W20260620 23:06:36.657119 1422 tablet.cc:2369] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:06:36.659601 1577 raft_consensus.cc:482] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:36.659725 1577 raft_consensus.cc:504] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:36.660121 1577 leader_election.cc:288] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015), b6b30fa096d74678a7e732e09451a96b (127.17.188.65:39691)
I20260620 23:06:36.662925 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.660258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:39691 (local address 127.17.188.66:44241)
0620 23:06:36.660304 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:36.660317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:36.660362 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:36.660544 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:36.660545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:36.660547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:36.660620 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.660621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.661319 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.661320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.662096 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.662098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.662666 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.662668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.662741 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.662745 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:06:36.662799 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 23:06:36.662856 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 23:06:36.662930 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.660368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:44241 (local address 127.17.188.65:39691)
0620 23:06:36.660413 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.660415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.660424 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:06:36.660455 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.660455 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.660469 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.660527 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:36.660699 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.661217 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.662189 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.662587 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.662672 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.662682 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.662816 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.662862 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:06:36.662999 1566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.660346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:39283 (local address 127.17.188.67:33015)
0620 23:06:36.660389 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.660391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.660527 (+ 136us) server_negotiation.cc:407] Connection header received
0620 23:06:36.660563 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.660564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.660572 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.660631 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:36.660800 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.661240 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.662216 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.662598 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.662705 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.662712 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.662891 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.662929 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:06:36.663046 1578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.660258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33015 (local address 127.17.188.66:39283)
0620 23:06:36.660440 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:36.660449 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:36.660512 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:36.660644 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:36.660644 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:36.660646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:36.660735 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.660737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.661400 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.661401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.662106 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:36.662107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.662705 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.662706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:36.662831 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.662834 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:06:36.662885 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 23:06:36.662920 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":9,"thread_start_us":108,"threads_started":1}
I20260620 23:06:36.663254 1241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54c3990805e4561b4d30fb8b253f42a" candidate_uuid: "9967ffca7fb3469ebd123c88ef441cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b30fa096d74678a7e732e09451a96b" is_pre_election: true
I20260620 23:06:36.663401 1241 raft_consensus.cc:2454] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9967ffca7fb3469ebd123c88ef441cde in term 0.
I20260620 23:06:36.663636 1310 leader_election.cc:302] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9967ffca7fb3469ebd123c88ef441cde, b6b30fa096d74678a7e732e09451a96b; no voters:
I20260620 23:06:36.663779 1577 raft_consensus.cc:2781] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:06:36.663837 1577 raft_consensus.cc:482] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:06:36.663762 1509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54c3990805e4561b4d30fb8b253f42a" candidate_uuid: "9967ffca7fb3469ebd123c88ef441cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" is_pre_election: true
I20260620 23:06:36.663854 1577 raft_consensus.cc:3037] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:36.663916 1509 raft_consensus.cc:2454] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9967ffca7fb3469ebd123c88ef441cde in term 0.
I20260620 23:06:36.664573 1577 raft_consensus.cc:504] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:36.664718 1577 leader_election.cc:288] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [CANDIDATE]: Term 1 election: Requested vote from peers 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015), b6b30fa096d74678a7e732e09451a96b (127.17.188.65:39691)
I20260620 23:06:36.664853 1509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54c3990805e4561b4d30fb8b253f42a" candidate_uuid: "9967ffca7fb3469ebd123c88ef441cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a"
I20260620 23:06:36.664870 1241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54c3990805e4561b4d30fb8b253f42a" candidate_uuid: "9967ffca7fb3469ebd123c88ef441cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b30fa096d74678a7e732e09451a96b"
I20260620 23:06:36.664942 1509 raft_consensus.cc:3037] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:36.664942 1241 raft_consensus.cc:3037] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:36.665822 1241 raft_consensus.cc:2454] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9967ffca7fb3469ebd123c88ef441cde in term 1.
I20260620 23:06:36.665822 1509 raft_consensus.cc:2454] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9967ffca7fb3469ebd123c88ef441cde in term 1.
I20260620 23:06:36.666002 1311 leader_election.cc:302] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c5fa5d8bf04c4e95fd886479ceb85a, 9967ffca7fb3469ebd123c88ef441cde; no voters:
I20260620 23:06:36.666110 1577 raft_consensus.cc:2781] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:36.666270 1577 raft_consensus.cc:686] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 1 LEADER]: Becoming Leader. State: Replica: 9967ffca7fb3469ebd123c88ef441cde, State: Running, Role: LEADER
I20260620 23:06:36.666371 1577 consensus_queue.cc:237] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:36.667153 1096 catalog_manager.cc:5495] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde reported cstate change: term changed from 0 to 1, leader changed from <none> to 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66). New cstate: current_term: 1 leader_uuid: "9967ffca7fb3469ebd123c88ef441cde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } health_report { overall_health: UNKNOWN } } }
I20260620 23:06:36.679473 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.677167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39852 (local address 127.17.188.66:34783)
0620 23:06:36.677220 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:36.677222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:36.677231 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:06:36.677282 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:36.677283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:36.677307 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:36.677421 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:06:36.677569 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.678016 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.678758 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:36.678941 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:36.679009 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:36.679174 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:06:36.679262 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:06:36.679356 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:36.679374 (+ 18us) server_negotiation.cc:299] Negotiation successful
0620 23:06:36.679392 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:06:36.680987 1509 raft_consensus.cc:1261] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [term 1 FOLLOWER]: Refusing update from remote peer 9967ffca7fb3469ebd123c88ef441cde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:36.681260 1241 raft_consensus.cc:1261] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Refusing update from remote peer 9967ffca7fb3469ebd123c88ef441cde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:36.681340 1579 consensus_queue.cc:1035] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [LEADER]: Connected to new peer: Peer: permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:36.681529 1577 consensus_queue.cc:1035] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:36.683364 1582 mvcc.cc:205] Tried to move back new op lower bound from 7299058879203233792 to 7299058879146074112. Current Snapshot: MvccSnapshot[applied={T|T < 7299058879203233792}]
I20260620 23:06:36.684713 1584 mvcc.cc:205] Tried to move back new op lower bound from 7299058879203233792 to 7299058879146074112. Current Snapshot: MvccSnapshot[applied={T|T < 7299058879203233792 or (T in {7299058879203233792})}]
I20260620 23:06:36.685595 1096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:36.685683 1096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:36.686329 1096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Sending DeleteTablet for 3 replicas of tablet d54c3990805e4561b4d30fb8b253f42a
I20260620 23:06:36.686651 1221 tablet_service.cc:1514] Processing DeleteTablet for tablet d54c3990805e4561b4d30fb8b253f42a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:36 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:33348
I20260620 23:06:36.686666 1489 tablet_service.cc:1514] Processing DeleteTablet for tablet d54c3990805e4561b4d30fb8b253f42a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:36 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:55746
I20260620 23:06:36.686858 1355 tablet_service.cc:1514] Processing DeleteTablet for tablet d54c3990805e4561b4d30fb8b253f42a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:36 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:39842
I20260620 23:06:36.686937 1592 tablet_replica.cc:330] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: stopping tablet replica
I20260620 23:06:36.686978 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 1158
I20260620 23:06:36.687040 1593 tablet_replica.cc:330] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: stopping tablet replica
I20260620 23:06:36.687839 1593 raft_consensus.cc:2229] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:06:36.687922 1593 raft_consensus.cc:2258] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:36.688294 1593 ts_tablet_manager.cc:1905] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:36.687023 1592 raft_consensus.cc:2229] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:06:36.689478 1592 raft_consensus.cc:2258] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:36.689870 1593 ts_tablet_manager.cc:1918] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:36.689957 1593 log.cc:1196] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/d54c3990805e4561b4d30fb8b253f42a
I20260620 23:06:36.689950 1592 ts_tablet_manager.cc:1905] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:36.690310 1593 ts_tablet_manager.cc:1939] T d54c3990805e4561b4d30fb8b253f42a P 9967ffca7fb3469ebd123c88ef441cde: Deleting consensus metadata
I20260620 23:06:36.690738 1081 catalog_manager.cc:4841] TS 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783): tablet d54c3990805e4561b4d30fb8b253f42a (table test-table [id=2266a2a528da466cbec2a043d9391977]) successfully deleted
I20260620 23:06:36.691576 1592 ts_tablet_manager.cc:1918] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:36.691687 1592 log.cc:1196] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/d54c3990805e4561b4d30fb8b253f42a
I20260620 23:06:36.691903 1592 ts_tablet_manager.cc:1939] T d54c3990805e4561b4d30fb8b253f42a P 16c5fa5d8bf04c4e95fd886479ceb85a: Deleting consensus metadata
I20260620 23:06:36.692243 1083 catalog_manager.cc:4841] TS 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015): tablet d54c3990805e4561b4d30fb8b253f42a (table test-table [id=2266a2a528da466cbec2a043d9391977]) successfully deleted
W20260620 23:06:36.692718 1082 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:39691 (error 108)
I20260620 23:06:36.693209 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:36.693075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:39691 (local address 127.0.0.1:33364)
0620 23:06:36.693124 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:36.693158 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:39691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 23:06:36.693279 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 1292
W20260620 23:06:36.694054 1082 catalog_manager.cc:4584] TS b6b30fa096d74678a7e732e09451a96b (127.17.188.65:39691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d54c3990805e4561b4d30fb8b253f42a: Network error: Client connection negotiation failed: client connection to 127.17.188.65:39691: connect: Connection refused (error 111)
I20260620 23:06:36.698884 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 1426
I20260620 23:06:36.703006 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 1064
I20260620 23:06:56.708279 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:40003
--webserver_interface=127.17.188.126
--webserver_port=38341
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:40003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:06:56.788705 1595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:56.788897 1595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:56.788919 1595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:56.790038 1595 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 23:06:56.790087 1595 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 23:06:56.790100 1595 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 23:06:56.790112 1595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:06:56.790187 1595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:56.790205 1595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:56.790231 1595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:06:56.790261 1595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:06:56.791668 1595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:40003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:40003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=38341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:56.791908 1595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:56.792165 1595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:06:56.800539 1595 server_base.cc:1034] running on GCE node
W20260620 23:06:56.800520 1601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:56.800513 1604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:56.800570 1602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:56.800981 1595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:56.801239 1595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:56.802419 1595 hybrid_clock.cc:648] HybridClock initialized: now 1781996816802385 us; error 40 us; skew 500 ppm
Jun 20 23:06:56 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996816, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:56.804883 1595 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:06:56.805255 1595 webserver.cc:460] Webserver started at http://127.17.188.126:38341/ using document root <none> and password file <none>
I20260620 23:06:56.805498 1595 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:56.805555 1595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:56.806819 1595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:56.807488 1611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:56.807708 1595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:56.807811 1595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "3d56aeed930749609d2d2df5fb33f1fa"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "82132cf4cb46fb00eac75c1f73fa65f7"
server_key_iv: "3427823e9f4ff023ab6dc152d139d5f5"
server_key_version: "encryptionkey@0"
I20260620 23:06:56.808214 1595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:56.828914 1595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:56.831104 1595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:56.831261 1595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:56.835233 1595 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:40003
I20260620 23:06:56.835279 1663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:40003 every 8 connection(s)
I20260620 23:06:56.835639 1595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:06:56.837038 1664 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:06:56.837796 1664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Bootstrap starting.
I20260620 23:06:56.843010 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 1595
I20260620 23:06:56.843305 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:39691
--local_ip_for_outbound_sockets=127.17.188.65
--tserver_master_addrs=127.17.188.126:40003
--webserver_port=39037
--webserver_interface=127.17.188.65
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:56.845901 1664 log.cc:824] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:56.848519 1664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:06:56.848845 1664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Bootstrap complete.
I20260620 23:06:56.852463 1664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:06:56.852818 1664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d56aeed930749609d2d2df5fb33f1fa, State: Initialized, Role: FOLLOWER
I20260620 23:06:56.852977 1664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:06:56.853053 1664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:06:56.853109 1664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:06:56.853163 1664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:06:56.854156 1664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:06:56.854240 1664 leader_election.cc:302] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d56aeed930749609d2d2df5fb33f1fa; no voters:
I20260620 23:06:56.854437 1664 leader_election.cc:288] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:06:56.854490 1668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:06:56.854624 1668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 2 LEADER]: Becoming Leader. State: Replica: 3d56aeed930749609d2d2df5fb33f1fa, State: Running, Role: LEADER
I20260620 23:06:56.854676 1664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:06:56.854799 1668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:06:56.855120 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3d56aeed930749609d2d2df5fb33f1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } } }
I20260620 23:06:56.855240 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:56.855144 1670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d56aeed930749609d2d2df5fb33f1fa. Latest consensus state: current_term: 2 leader_uuid: "3d56aeed930749609d2d2df5fb33f1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } } }
I20260620 23:06:56.855405 1670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: This master's current role is: LEADER
I20260620 23:06:56.855952 1676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:06:56.856680 1676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:06:56.856803 1676 catalog_manager.cc:1259] Loaded cluster ID: 6cae1c575858489c93d7afb64519bfbd
I20260620 23:06:56.856870 1676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:06:56.857734 1676 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:06:56.857908 1676 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Loaded TSK: 0
I20260620 23:06:56.867949 1676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Generated new TSK 1
I20260620 23:06:56.868060 1676 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:06:56.935602 1666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:56.935801 1666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:56.935834 1666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:56.936926 1666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:56.936983 1666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:56.937026 1666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:06:56.938485 1666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:39691
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=39037
--tserver_master_addrs=127.17.188.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:56.938711 1666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:56.938901 1666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:56.944366 1692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:56.944447 1691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:56.944554 1666 server_base.cc:1034] running on GCE node
W20260620 23:06:56.944693 1694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:56.944892 1666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:56.945115 1666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:56.946285 1666 hybrid_clock.cc:648] HybridClock initialized: now 1781996816946259 us; error 43 us; skew 500 ppm
Jun 20 23:06:56 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996816, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:56.948781 1666 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:06:56.949270 1666 webserver.cc:460] Webserver started at http://127.17.188.65:39037/ using document root <none> and password file <none>
I20260620 23:06:56.949518 1666 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:56.949573 1666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:56.950838 1666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:56.951508 1701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:56.951666 1666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:56.951752 1666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "b6b30fa096d74678a7e732e09451a96b"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "09dec22912f5d7578d8d11c2c8f4a745"
server_key_iv: "60bb287d6c170806a333105cabb9bc22"
server_key_version: "encryptionkey@0"
I20260620 23:06:56.952118 1666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:56.970211 1666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:56.972395 1666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:56.972541 1666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:56.972762 1666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:56.973206 1708 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 23:06:56.974550 1666 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 23:06:56.974607 1666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 23:06:56.974640 1666 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 23:06:56.975433 1666 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 23:06:56.975478 1666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 23:06:56.975544 1708 tablet_bootstrap.cc:492] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Bootstrap starting.
I20260620 23:06:56.982266 1666 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:39691
I20260620 23:06:56.982359 1815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:39691 every 8 connection(s)
I20260620 23:06:56.982704 1666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:06:56.985994 1708 log.cc:824] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:56.988183 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 1666
I20260620 23:06:56.988446 1708 tablet_bootstrap.cc:492] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:06:56.988510 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:34783
--local_ip_for_outbound_sockets=127.17.188.66
--tserver_master_addrs=127.17.188.126:40003
--webserver_port=36491
--webserver_interface=127.17.188.66
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:56.988732 1708 tablet_bootstrap.cc:492] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Bootstrap complete.
I20260620 23:06:56.989883 1708 ts_tablet_manager.cc:1397] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.007s
I20260620 23:06:56.991398 1708 raft_consensus.cc:348] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:56.991761 1708 raft_consensus.cc:729] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b30fa096d74678a7e732e09451a96b, State: Initialized, Role: FOLLOWER
Jun 20 23:06:56 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996816, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:56.991921 1708 consensus_queue.cc:260] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:56.992219 1708 ts_tablet_manager.cc:1428] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260620 23:06:56.996603 1818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:56.983524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:58913)
0620 23:06:56.983735 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:56.983783 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:56.984824 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:56.985370 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:56.985376 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:56.985751 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:56.986026 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:56.986032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:56.987681 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:56.987682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:56.988977 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:56.988980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:56.989904 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:56.989906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:56.990042 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:56.990659 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:56.990669 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:56.992297 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:56.994735 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:56.994741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:56.994744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:56.995025 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:56.995325 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:56.995326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:56.995326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:56.995468 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:56.995968 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:56.995976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:56.996120 (+ 144us) client_negotiation.cc:769] Sending connection context
0620 23:06:56.996245 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 23:06:56.996411 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":98,"threads_started":1}
I20260620 23:06:56.996907 1819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:56.983860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:58913 (local address 127.17.188.126:40003)
0620 23:06:56.984051 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:56.984060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:56.984101 (+ 41us) server_negotiation.cc:407] Connection header received
0620 23:06:56.985064 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:56.985071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:56.985127 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:56.985280 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:56.986167 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:56.987546 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:56.989079 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:56.989719 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:56.989815 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:56.992421 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:56.992425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:56.992425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:56.992438 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:56.994645 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:56.995115 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:56.995118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:56.995119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:56.995243 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:56.995551 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:56.995552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:56.995552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:56.995882 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:56.995973 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:56.996327 (+ 354us) server_negotiation.cc:299] Negotiation successful
0620 23:06:56.996473 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260620 23:06:56.997496 1816 heartbeater.cc:347] Connected to a master server at 127.17.188.126:40003
I20260620 23:06:56.997601 1816 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:56.997802 1816 heartbeater.cc:511] Master 127.17.188.126:40003 requested a full tablet report, sending...
I20260620 23:06:56.998593 1628 ts_manager.cc:194] Registered new tserver with Master: b6b30fa096d74678a7e732e09451a96b (127.17.188.65:39691)
I20260620 23:06:56.999634 1628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:58913
I20260620 23:06:57.000525 1816 heartbeater.cc:503] Master 127.17.188.126:40003 was elected leader, sending a full tablet report...
Jun 20 23:06:56 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996816, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.65@KRBTEST.COM
I20260620 23:06:57.008599 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:56.999388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.17.188.65:39691)
0620 23:06:56.999768 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:56.999773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:56.999781 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:06:56.999813 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:56.999814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.000041 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.000110 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:57.000683 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.001899 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.002984 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.003438 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.003514 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.005869 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:57.005877 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:57.005878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:57.005885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:57.007437 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.007865 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.007867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.007868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.007924 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.008117 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.008118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.008118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.008252 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:57.008309 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.008473 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.008539 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":63,"threads_started":1}
I20260620 23:06:57.008656 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:56.999149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:39691 (local address 127.0.0.1:53214)
0620 23:06:56.999534 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:56.999572 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:56.999637 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:57.000173 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:57.000179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:57.000528 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:57.000589 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.000591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.001987 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.001988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.002892 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.002895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.003532 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.003533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.003604 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.004122 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:57.004127 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:57.005775 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:57.007521 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:57.007529 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:57.007531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:57.007785 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:57.007999 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:57.008000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:57.008000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:57.008061 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:57.008312 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:57.008317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:57.008367 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 23:06:57.008453 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:06:57.008558 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":81,"threads_started":1}
I20260620 23:06:57.009150 1735 tablet_service.cc:1514] Processing DeleteTablet for tablet d54c3990805e4561b4d30fb8b253f42a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:36 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:53214
I20260620 23:06:57.009720 1825 tablet_replica.cc:330] stopping tablet replica
I20260620 23:06:57.009828 1825 raft_consensus.cc:2229] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:06:57.009895 1825 raft_consensus.cc:2258] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:57.010181 1825 ts_tablet_manager.cc:1905] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:57.012274 1825 ts_tablet_manager.cc:1918] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:57.012332 1825 log.cc:1196] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal/wals/d54c3990805e4561b4d30fb8b253f42a
I20260620 23:06:57.012558 1825 ts_tablet_manager.cc:1939] T d54c3990805e4561b4d30fb8b253f42a P b6b30fa096d74678a7e732e09451a96b: Deleting consensus metadata
I20260620 23:06:57.013046 1614 catalog_manager.cc:4841] TS b6b30fa096d74678a7e732e09451a96b (127.17.188.65:39691): tablet d54c3990805e4561b4d30fb8b253f42a (table test-table [id=2266a2a528da466cbec2a043d9391977]) successfully deleted
W20260620 23:06:57.073385 1821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:57.073599 1821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:57.073637 1821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:57.074782 1821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:57.074846 1821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:57.074892 1821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:06:57.076299 1821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:34783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=36491
--tserver_master_addrs=127.17.188.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:57.076550 1821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:57.076763 1821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:57.081503 1833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:57.081529 1830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:57.081568 1821 server_base.cc:1034] running on GCE node
W20260620 23:06:57.081522 1831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:57.081975 1821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:57.082312 1821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:57.083467 1821 hybrid_clock.cc:648] HybridClock initialized: now 1781996817083452 us; error 29 us; skew 500 ppm
Jun 20 23:06:57 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996817, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:57.085971 1821 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:06:57.086412 1821 webserver.cc:460] Webserver started at http://127.17.188.66:36491/ using document root <none> and password file <none>
I20260620 23:06:57.086635 1821 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:57.086685 1821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:57.087946 1821 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:57.088537 1840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:57.088779 1821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:06:57.088845 1821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "9967ffca7fb3469ebd123c88ef441cde"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "8fbf4aadff7ea7a9eaa3397d2e16f19d"
server_key_iv: "778fe534dc013b1f0af872e309435468"
server_key_version: "encryptionkey@0"
I20260620 23:06:57.089139 1821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:57.117723 1821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:57.120019 1821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:57.120183 1821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:57.120411 1821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:57.120764 1821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:57.120810 1821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:57.120847 1821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:57.120875 1821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:57.126797 1821 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:34783
I20260620 23:06:57.126853 1953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:34783 every 8 connection(s)
I20260620 23:06:57.127169 1821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:06:57 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996817, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:57.135737 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 1821
I20260620 23:06:57.136027 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:33015
--local_ip_for_outbound_sockets=127.17.188.67
--tserver_master_addrs=127.17.188.126:40003
--webserver_port=33701
--webserver_interface=127.17.188.67
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:06:57.136097 1819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.127849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:42539 (local address 127.17.188.126:40003)
0620 23:06:57.127914 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.127916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.128177 (+ 261us) server_negotiation.cc:407] Connection header received
0620 23:06:57.128934 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.128936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.128949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.129002 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:57.129987 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.130467 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.131373 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.131559 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.131607 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.133611 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:57.133612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:57.133612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:57.133618 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:57.134868 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.135227 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.135228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.135229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.135275 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.135470 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.135471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.135471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.135581 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:57.135627 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.135954 (+ 327us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.136017 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:06:57.136353 1956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.127829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:42539)
0620 23:06:57.127978 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:57.128044 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:57.128829 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:57.129069 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:57.129072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:57.129719 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:57.129903 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.129907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.130540 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.130541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.131301 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.131303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.131651 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.131652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.131748 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.132172 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:57.132180 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:57.133534 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:57.134938 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:57.134943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:57.134945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:57.135165 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:57.135328 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:57.135329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:57.135329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:57.135419 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:57.135652 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:57.135658 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:57.135815 (+ 157us) client_negotiation.cc:769] Sending connection context
0620 23:06:57.135935 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 23:06:57.136096 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260620 23:06:57.136878 1954 heartbeater.cc:347] Connected to a master server at 127.17.188.126:40003
I20260620 23:06:57.136978 1954 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:57.137161 1954 heartbeater.cc:511] Master 127.17.188.126:40003 requested a full tablet report, sending...
I20260620 23:06:57.137611 1628 ts_manager.cc:194] Registered new tserver with Master: 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783)
I20260620 23:06:57.137979 1628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:42539
W20260620 23:06:57.215161 1957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:06:57.215368 1957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:06:57.215401 1957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:06:57.216472 1957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:06:57.216532 1957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:06:57.216572 1957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:06:57.218001 1957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:33015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=33701
--tserver_master_addrs=127.17.188.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:06:57.218258 1957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:06:57.218465 1957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:06:57.223150 1965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:57.223172 1962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:06:57.223380 1963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:06:57.223354 1957 server_base.cc:1034] running on GCE node
I20260620 23:06:57.223647 1957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:06:57.223877 1957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:06:57.225032 1957 hybrid_clock.cc:648] HybridClock initialized: now 1781996817225017 us; error 30 us; skew 500 ppm
Jun 20 23:06:57 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996817, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:06:57.227453 1957 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:06:57.227880 1957 webserver.cc:460] Webserver started at http://127.17.188.67:33701/ using document root <none> and password file <none>
I20260620 23:06:57.228111 1957 fs_manager.cc:359] Metadata directory not provided
I20260620 23:06:57.228188 1957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:06:57.229442 1957 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:06:57.230001 1972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:57.230170 1957 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:06:57.230258 1957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "16c5fa5d8bf04c4e95fd886479ceb85a"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "a0d5e0fee8ba2c1e6db7f5da154c4aab"
server_key_iv: "2bff0c579add74c2d914b7b9483afeec"
server_key_version: "encryptionkey@0"
I20260620 23:06:57.230554 1957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:06:57.253703 1957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:06:57.256266 1957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:06:57.256430 1957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:06:57.256675 1957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:06:57.257014 1957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:06:57.257051 1957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:57.257076 1957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:06:57.257134 1957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:06:57.263478 1957 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:33015
I20260620 23:06:57.263545 2085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:33015 every 8 connection(s)
I20260620 23:06:57.263881 1957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:06:57 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996817, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:06:57.271329 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 1957
I20260620 23:06:57.272735 1819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.264559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:50039 (local address 127.17.188.126:40003)
0620 23:06:57.264656 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.264658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.264884 (+ 226us) server_negotiation.cc:407] Connection header received
0620 23:06:57.265296 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.265297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.265307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.265444 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:57.266271 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.266736 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.267677 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.267862 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.267918 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.269832 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:57.269834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:57.269834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:57.269840 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:57.271099 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.271622 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.271623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.271624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.271718 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.272031 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.272032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.272033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.272205 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:57.272279 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.272549 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.272631 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 23:06:57.272837 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.264549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:50039)
0620 23:06:57.264742 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:57.264780 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:57.265197 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:57.265495 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:57.265498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:57.266002 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:06:57.266163 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.266167 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.266809 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.266810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.267578 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.267580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.267935 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.267936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.268030 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.268398 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:06:57.268405 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:06:57.269713 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:06:57.271210 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:57.271235 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:57.271237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:57.271540 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:57.271817 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:06:57.271818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:06:57.271818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:06:57.271954 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:06:57.272293 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:06:57.272298 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:06:57.272407 (+ 109us) client_negotiation.cc:769] Sending connection context
0620 23:06:57.272499 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 23:06:57.272611 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260620 23:06:57.273212 2086 heartbeater.cc:347] Connected to a master server at 127.17.188.126:40003
I20260620 23:06:57.273365 2086 heartbeater.cc:464] Registering TS with master...
I20260620 23:06:57.273555 2086 heartbeater.cc:511] Master 127.17.188.126:40003 requested a full tablet report, sending...
I20260620 23:06:57.273887 1628 ts_manager.cc:194] Registered new tserver with Master: 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015)
I20260620 23:06:57.274359 1628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:50039
I20260620 23:06:57.278757 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.271791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54568 (local address 127.17.188.126:40003)
0620 23:06:57.271929 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.271931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.271999 (+ 68us) server_negotiation.cc:407] Connection header received
0620 23:06:57.272062 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.272063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.272074 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.272129 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:57.273209 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.273844 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.274900 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.275166 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.275222 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.276316 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:57.276317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:57.276317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:57.276322 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:57.277841 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.278130 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.278131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.278132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.278196 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.278377 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.278377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.278378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.278466 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:57.278571 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.278640 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.278699 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260620 23:06:57.279456 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:06:57.281898 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.279807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54582 (local address 127.17.188.126:40003)
0620 23:06:57.279872 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.279880 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.279890 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:06:57.279970 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.279971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.279997 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.280052 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:06:57.280124 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.280540 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.281185 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.281416 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.281474 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.281624 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:06:57.281687 (+ 63us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 23:06:57.281831 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54582: token expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 23:06:57.282001 2090 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54582: token expired
W20260620 23:06:57.281992 1561 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.188.126:40003)
I20260620 23:06:57.287339 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.282228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54586 (local address 127.17.188.126:40003)
0620 23:06:57.282263 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.282264 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.282302 (+ 38us) server_negotiation.cc:407] Connection header received
0620 23:06:57.282359 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.282359 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.282366 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.282415 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:06:57.282845 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.283244 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.283912 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.284072 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.284117 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.285174 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:06:57.285175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:06:57.285176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:06:57.285181 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:06:57.286493 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.286821 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.286821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.286822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.286882 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:06:57.287009 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:06:57.287009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:06:57.287009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:06:57.287085 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:06:57.287145 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.287243 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.287291 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:06:57.288359 1628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:06:57.289971 1628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:06:57.292174 1735 tablet_service.cc:1467] Processing CreateTablet for tablet 19256433baf740b899aebff7393f9d38 (DEFAULT_TABLE table=test-table [id=f7eb3fa31d2c4c41a15993d887bc721f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:57.292384 1735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19256433baf740b899aebff7393f9d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:57.294147 1708 tablet_bootstrap.cc:492] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Bootstrap starting.
I20260620 23:06:57.294507 1708 tablet_bootstrap.cc:654] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:57.295094 1708 tablet_bootstrap.cc:492] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: No bootstrap required, opened a new log
I20260620 23:06:57.295166 1708 ts_tablet_manager.cc:1397] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 23:06:57.295449 2092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.291889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33015 (local address 127.0.0.1:47796)
0620 23:06:57.292035 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:57.292044 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:57.292087 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:57.292441 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:57.292442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:57.292444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:57.292484 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.292486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.293496 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.293497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.294276 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.294277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.295247 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.295248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.295307 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.295328 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 23:06:57.295358 (+ 30us) client_negotiation.cc:240] Negotiation successful
0620 23:06:57.295395 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260620 23:06:57.295545 1708 raft_consensus.cc:348] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:57.295644 1708 raft_consensus.cc:374] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:57.295692 1708 raft_consensus.cc:729] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b30fa096d74678a7e732e09451a96b, State: Initialized, Role: FOLLOWER
I20260620 23:06:57.295742 1708 consensus_queue.cc:260] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:57.295786 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.291991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.17.188.67:33015)
0620 23:06:57.292186 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.292190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.292202 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:06:57.292239 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.292241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.292256 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.292343 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:57.292560 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.293417 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.294374 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.295158 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.295572 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.295635 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.295660 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.295705 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260620 23:06:57.295845 1708 ts_tablet_manager.cc:1428] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 23:06:57.296545 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.291825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:34783 (local address 127.0.0.1:47902)
0620 23:06:57.291889 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:57.291900 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:57.291937 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:57.292850 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:57.292851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:57.292852 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:57.292889 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.292890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.294240 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.294241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.295276 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.295277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.296323 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.296324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.296388 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.296391 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:06:57.296447 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 23:06:57.296491 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 23:06:57.296747 2020 tablet_service.cc:1467] Processing CreateTablet for tablet 19256433baf740b899aebff7393f9d38 (DEFAULT_TABLE table=test-table [id=f7eb3fa31d2c4c41a15993d887bc721f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:57.296857 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.292028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.17.188.66:34783)
0620 23:06:57.292669 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.292673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.292684 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:06:57.292725 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.292727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.292742 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.292969 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:57.293084 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.294139 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.295356 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.296057 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.296166 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.296222 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.296744 (+ 522us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.296800 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":94,"threads_started":1}
I20260620 23:06:57.297097 2020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19256433baf740b899aebff7393f9d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:57.297461 1888 tablet_service.cc:1467] Processing CreateTablet for tablet 19256433baf740b899aebff7393f9d38 (DEFAULT_TABLE table=test-table [id=f7eb3fa31d2c4c41a15993d887bc721f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:06:57.297735 1888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19256433baf740b899aebff7393f9d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:06:57.299765 2096 tablet_bootstrap.cc:492] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: Bootstrap starting.
I20260620 23:06:57.300124 2097 tablet_bootstrap.cc:492] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: Bootstrap starting.
I20260620 23:06:57.300426 2096 tablet_bootstrap.cc:654] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:57.300665 2097 tablet_bootstrap.cc:654] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: Neither blocks nor log segments found. Creating new log.
I20260620 23:06:57.300972 2096 log.cc:824] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:57.301136 2097 log.cc:824] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: Log is configured to *not* fsync() on all Append() calls
I20260620 23:06:57.301805 2096 tablet_bootstrap.cc:492] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: No bootstrap required, opened a new log
I20260620 23:06:57.301872 2096 ts_tablet_manager.cc:1397] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:06:57.301908 2097 tablet_bootstrap.cc:492] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: No bootstrap required, opened a new log
I20260620 23:06:57.301966 2097 ts_tablet_manager.cc:1397] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:06:57.303795 2096 raft_consensus.cc:348] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:57.303836 2097 raft_consensus.cc:348] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:57.303974 2097 raft_consensus.cc:374] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:57.303974 2096 raft_consensus.cc:374] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:06:57.304009 2097 raft_consensus.cc:729] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9967ffca7fb3469ebd123c88ef441cde, State: Initialized, Role: FOLLOWER
I20260620 23:06:57.304009 2096 raft_consensus.cc:729] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c5fa5d8bf04c4e95fd886479ceb85a, State: Initialized, Role: FOLLOWER
I20260620 23:06:57.304114 2096 consensus_queue.cc:260] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:57.304119 2097 consensus_queue.cc:260] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:57.304396 2096 ts_tablet_manager.cc:1428] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:06:57.304396 2097 ts_tablet_manager.cc:1428] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 23:06:57.304426 2086 heartbeater.cc:503] Master 127.17.188.126:40003 was elected leader, sending a full tablet report...
I20260620 23:06:57.304512 1954 heartbeater.cc:503] Master 127.17.188.126:40003 was elected leader, sending a full tablet report...
W20260620 23:06:57.377676 1955 tablet.cc:2369] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:06:57.452373 2100 raft_consensus.cc:482] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:06:57.452554 2100 raft_consensus.cc:504] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:57.452986 2100 leader_election.cc:288] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783), b6b30fa096d74678a7e732e09451a96b (127.17.188.65:39691)
I20260620 23:06:57.456372 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.453124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:34783 (local address 127.17.188.67:37689)
0620 23:06:57.453179 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:57.453190 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:57.453250 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:57.453476 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:57.453477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:57.453479 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:57.453555 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.453556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.454456 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.454456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.455423 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.455426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.456071 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.456072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.456149 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.456151 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:06:57.456212 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:06:57.456285 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":1}
I20260620 23:06:57.456660 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.453219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:37689 (local address 127.17.188.66:34783)
0620 23:06:57.453289 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.453291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.453303 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:06:57.453365 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.453366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.453383 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.453443 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:57.453624 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.454381 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.455516 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.456013 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.456075 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.456083 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.456562 (+ 479us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.456602 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:06:57.457005 1908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19256433baf740b899aebff7393f9d38" candidate_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9967ffca7fb3469ebd123c88ef441cde" is_pre_election: true
I20260620 23:06:57.457152 1908 raft_consensus.cc:2454] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16c5fa5d8bf04c4e95fd886479ceb85a in term 0.
I20260620 23:06:57.457429 1974 leader_election.cc:302] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c5fa5d8bf04c4e95fd886479ceb85a, 9967ffca7fb3469ebd123c88ef441cde; no voters:
I20260620 23:06:57.457686 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.453176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:39691 (local address 127.17.188.67:55941)
0620 23:06:57.453371 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 23:06:57.453379 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:06:57.453457 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:06:57.453672 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:06:57.453672 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:06:57.453673 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:06:57.453722 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.453723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.454573 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.454575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.455378 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:06:57.455381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.456180 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.456181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:06:57.457452 (+ 1271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.457457 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:06:57.457583 (+ 126us) client_negotiation.cc:240] Negotiation successful
0620 23:06:57.457622 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":111,"threads_started":1}
I20260620 23:06:57.457712 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.453280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:55941 (local address 127.17.188.65:39691)
0620 23:06:57.453337 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.453339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.453444 (+ 105us) server_negotiation.cc:407] Connection header received
0620 23:06:57.453540 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.453541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.453563 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.453658 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:06:57.453790 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.454476 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.455480 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.456056 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.456180 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.456273 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.457597 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.457639 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:06:57.458134 1752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19256433baf740b899aebff7393f9d38" candidate_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b30fa096d74678a7e732e09451a96b" is_pre_election: true
I20260620 23:06:57.458278 1752 raft_consensus.cc:2454] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16c5fa5d8bf04c4e95fd886479ceb85a in term 0.
I20260620 23:06:57.461232 2100 raft_consensus.cc:2781] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:06:57.461336 2100 raft_consensus.cc:482] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:06:57.461367 2100 raft_consensus.cc:3037] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:57.462167 2100 raft_consensus.cc:504] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:57.462311 2100 leader_election.cc:288] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [CANDIDATE]: Term 1 election: Requested vote from peers 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783), b6b30fa096d74678a7e732e09451a96b (127.17.188.65:39691)
I20260620 23:06:57.462652 1752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19256433baf740b899aebff7393f9d38" candidate_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b30fa096d74678a7e732e09451a96b"
I20260620 23:06:57.462769 1752 raft_consensus.cc:3037] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:57.463678 1752 raft_consensus.cc:2454] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16c5fa5d8bf04c4e95fd886479ceb85a in term 1.
I20260620 23:06:57.463945 1975 leader_election.cc:302] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c5fa5d8bf04c4e95fd886479ceb85a, b6b30fa096d74678a7e732e09451a96b; no voters:
I20260620 23:06:57.463946 1908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19256433baf740b899aebff7393f9d38" candidate_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9967ffca7fb3469ebd123c88ef441cde"
I20260620 23:06:57.464025 2100 raft_consensus.cc:2781] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:06:57.464036 1908 raft_consensus.cc:3037] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:06:57.464205 2100 raft_consensus.cc:686] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 1 LEADER]: Becoming Leader. State: Replica: 16c5fa5d8bf04c4e95fd886479ceb85a, State: Running, Role: LEADER
I20260620 23:06:57.464303 2100 consensus_queue.cc:237] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:06:57.464939 1908 raft_consensus.cc:2454] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16c5fa5d8bf04c4e95fd886479ceb85a in term 1.
I20260620 23:06:57.465469 1627 catalog_manager.cc:5495] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a reported cstate change: term changed from 0 to 1, leader changed from <none> to 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67). New cstate: current_term: 1 leader_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } health_report { overall_health: UNKNOWN } } }
I20260620 23:06:57.475785 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:06:57.473645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.17.188.67:33015)
0620 23:06:57.473694 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:06:57.473696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:06:57.473728 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:06:57.473796 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:06:57.473797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:06:57.473810 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:06:57.473865 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:06:57.473961 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.474409 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.475169 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:06:57.475335 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:06:57.475379 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:06:57.475500 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:06:57.475578 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:06:57.475633 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:06:57.475717 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 23:06:57.475730 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 23:06:57.477213 1752 raft_consensus.cc:1261] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Refusing update from remote peer 16c5fa5d8bf04c4e95fd886479ceb85a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:57.477214 1908 raft_consensus.cc:1261] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [term 1 FOLLOWER]: Refusing update from remote peer 16c5fa5d8bf04c4e95fd886479ceb85a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:06:57.477556 2103 consensus_queue.cc:1035] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:57.477685 2100 consensus_queue.cc:1035] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:06:57.479879 2106 mvcc.cc:205] Tried to move back new op lower bound from 7299058964384456704 to 7299058964334374912. Current Snapshot: MvccSnapshot[applied={T|T < 7299058964384456704 or (T in {7299058964384456704})}]
I20260620 23:06:57.481464 1627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:57.481564 1627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:06:57.482200 1627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Sending DeleteTablet for 3 replicas of tablet 19256433baf740b899aebff7393f9d38
I20260620 23:06:57.482491 1735 tablet_service.cc:1514] Processing DeleteTablet for tablet 19256433baf740b899aebff7393f9d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:57 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:53214
I20260620 23:06:57.482499 1888 tablet_service.cc:1514] Processing DeleteTablet for tablet 19256433baf740b899aebff7393f9d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:57 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:47902
I20260620 23:06:57.482625 1825 tablet_replica.cc:330] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: stopping tablet replica
I20260620 23:06:57.482695 1825 raft_consensus.cc:2229] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:06:57.482709 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 1666
I20260620 23:06:57.482743 1825 raft_consensus.cc:2258] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:57.483017 1825 ts_tablet_manager.cc:1905] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:57.485286 2020 tablet_service.cc:1514] Processing DeleteTablet for tablet 19256433baf740b899aebff7393f9d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:06:57 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:47796
I20260620 23:06:57.485495 2117 tablet_replica.cc:330] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: stopping tablet replica
I20260620 23:06:57.485514 2116 tablet_replica.cc:330] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: stopping tablet replica
I20260620 23:06:57.485582 2117 raft_consensus.cc:2229] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:06:57.485664 2117 raft_consensus.cc:2258] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:57.485695 2116 raft_consensus.cc:2229] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:06:57.485874 2116 raft_consensus.cc:2258] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:06:57.486043 2117 ts_tablet_manager.cc:1905] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:57.486351 2116 ts_tablet_manager.cc:1905] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:06:57.487900 2116 ts_tablet_manager.cc:1918] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:57.488005 2116 log.cc:1196] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/19256433baf740b899aebff7393f9d38
I20260620 23:06:57.488205 2117 ts_tablet_manager.cc:1918] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:06:57.488322 2117 log.cc:1196] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/19256433baf740b899aebff7393f9d38
I20260620 23:06:57.488408 2116 ts_tablet_manager.cc:1939] T 19256433baf740b899aebff7393f9d38 P 9967ffca7fb3469ebd123c88ef441cde: Deleting consensus metadata
I20260620 23:06:57.488631 2117 ts_tablet_manager.cc:1939] T 19256433baf740b899aebff7393f9d38 P 16c5fa5d8bf04c4e95fd886479ceb85a: Deleting consensus metadata
I20260620 23:06:57.488858 1613 catalog_manager.cc:4841] TS 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783): tablet 19256433baf740b899aebff7393f9d38 (table test-table [id=f7eb3fa31d2c4c41a15993d887bc721f]) successfully deleted
I20260620 23:06:57.489102 1615 catalog_manager.cc:4841] TS 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015): tablet 19256433baf740b899aebff7393f9d38 (table test-table [id=f7eb3fa31d2c4c41a15993d887bc721f]) successfully deleted
W20260620 23:06:57.489284 1614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:39691 (error 108)
I20260620 23:06:57.489943 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 1821
I20260620 23:06:57.494511 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 1957
I20260620 23:06:57.499472 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 1595
I20260620 23:07:17.504030 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:40003
--webserver_interface=127.17.188.126
--webserver_port=38341
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:40003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:17.581897 2119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:17.582077 2119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:17.582098 2119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:17.583276 2119 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 23:07:17.583314 2119 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 23:07:17.583328 2119 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 23:07:17.583345 2119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:17.583391 2119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:17.583417 2119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:17.583442 2119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:17.583470 2119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:17.584923 2119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:40003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:40003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=38341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:17.585140 2119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:17.585414 2119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:17.590003 2125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:17.590019 2119 server_base.cc:1034] running on GCE node
W20260620 23:07:17.590000 2127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:17.590000 2124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:17.590461 2119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:17.590776 2119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:17.591914 2119 hybrid_clock.cc:648] HybridClock initialized: now 1781996837591902 us; error 28 us; skew 500 ppm
Jun 20 23:07:17 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996837, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:17.594275 2119 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:07:17.594584 2119 webserver.cc:460] Webserver started at http://127.17.188.126:38341/ using document root <none> and password file <none>
I20260620 23:07:17.594772 2119 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:17.594810 2119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:17.595988 2119 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:07:17.596560 2134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:17.596763 2119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:17.596864 2119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "3d56aeed930749609d2d2df5fb33f1fa"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "82132cf4cb46fb00eac75c1f73fa65f7"
server_key_iv: "3427823e9f4ff023ab6dc152d139d5f5"
server_key_version: "encryptionkey@0"
I20260620 23:07:17.597190 2119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:17.609159 2119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:17.610862 2119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:17.611014 2119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:17.614663 2119 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:40003
I20260620 23:07:17.614717 2186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:40003 every 8 connection(s)
I20260620 23:07:17.615057 2119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:17.616343 2187 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:07:17.616878 2187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Bootstrap starting.
I20260620 23:07:17.618726 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 2119
I20260620 23:07:17.619053 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:39691
--local_ip_for_outbound_sockets=127.17.188.65
--tserver_master_addrs=127.17.188.126:40003
--webserver_port=39037
--webserver_interface=127.17.188.65
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:07:17.625640 2187 log.cc:824] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:17.627705 2187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:07:17.627990 2187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Bootstrap complete.
I20260620 23:07:17.630237 2187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:07:17.630458 2187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d56aeed930749609d2d2df5fb33f1fa, State: Initialized, Role: FOLLOWER
I20260620 23:07:17.630625 2187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:07:17.630681 2187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:17.630740 2187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:17.630790 2187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 2 FOLLOWER]: Advancing to term 3
I20260620 23:07:17.631767 2187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:07:17.631870 2187 leader_election.cc:302] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d56aeed930749609d2d2df5fb33f1fa; no voters:
I20260620 23:07:17.632062 2187 leader_election.cc:288] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 23:07:17.632098 2191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 3 FOLLOWER]: Leader election won for term 3
I20260620 23:07:17.632284 2191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [term 3 LEADER]: Becoming Leader. State: Replica: 3d56aeed930749609d2d2df5fb33f1fa, State: Running, Role: LEADER
I20260620 23:07:17.632339 2187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:17.632437 2191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } }
I20260620 23:07:17.632839 2192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3d56aeed930749609d2d2df5fb33f1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } } }
I20260620 23:07:17.632929 2192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:17.633230 2199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:17.633350 2193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d56aeed930749609d2d2df5fb33f1fa. Latest consensus state: current_term: 3 leader_uuid: "3d56aeed930749609d2d2df5fb33f1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d56aeed930749609d2d2df5fb33f1fa" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 40003 } } }
I20260620 23:07:17.633504 2193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:17.634096 2199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:17.634393 2199 catalog_manager.cc:1259] Loaded cluster ID: 6cae1c575858489c93d7afb64519bfbd
I20260620 23:07:17.634428 2199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:17.635536 2199 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:17.635695 2199 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Loaded TSK: 0
I20260620 23:07:17.635735 2199 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3d56aeed930749609d2d2df5fb33f1fa: Loaded TSK: 1
F20260620 23:07:17.635952 2199 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f28997299ad google::LogMessage::Fail() at ??:0
@ 0x7f289972c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f28997294de google::LogMessage::Flush() at ??:0
@ 0x7f289972aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f289be96104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f289be93d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f28a16831cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f28a168348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f28a1683594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f28a1649c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f28a1653ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f289b5f70d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f289b5ec954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f289c4506db start_thread at ??:0
@ 0x7f289822761f clone at ??:0
W20260620 23:07:17.712764 2189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:17.712965 2189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:17.713001 2189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:17.714250 2189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:17.714311 2189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:17.714354 2189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:07:17.715703 2189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:39691
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=39037
--tserver_master_addrs=127.17.188.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:17.715929 2189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:17.716120 2189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:17.721750 2229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:17.721798 2189 server_base.cc:1034] running on GCE node
W20260620 23:07:17.721920 2228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:17.721740 2231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:17.722178 2189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:17.722435 2189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:17.723583 2189 hybrid_clock.cc:648] HybridClock initialized: now 1781996837723568 us; error 28 us; skew 500 ppm
Jun 20 23:07:17 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996837, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:17.726298 2189 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:07:17.726702 2189 webserver.cc:460] Webserver started at http://127.17.188.65:39037/ using document root <none> and password file <none>
I20260620 23:07:17.726900 2189 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:17.726960 2189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:17.728243 2189 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:07:17.728931 2238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:17.729085 2189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:17.729175 2189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "b6b30fa096d74678a7e732e09451a96b"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "09dec22912f5d7578d8d11c2c8f4a745"
server_key_iv: "60bb287d6c170806a333105cabb9bc22"
server_key_version: "encryptionkey@0"
I20260620 23:07:17.729521 2189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:17.742950 2189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:17.745142 2189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:17.745282 2189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:17.745498 2189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:17.745893 2245 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 23:07:17.747747 2189 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 23:07:17.747802 2189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 23:07:17.747841 2189 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 23:07:17.748413 2189 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 23:07:17.748481 2189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 23:07:17.748558 2245 tablet_bootstrap.cc:492] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Bootstrap starting.
I20260620 23:07:17.750391 2245 log.cc:824] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:17.753137 2245 tablet_bootstrap.cc:492] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:07:17.753502 2245 tablet_bootstrap.cc:492] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Bootstrap complete.
I20260620 23:07:17.753690 2245 ts_tablet_manager.cc:1397] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260620 23:07:17.755522 2189 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:39691
I20260620 23:07:17.755569 2353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:39691 every 8 connection(s)
I20260620 23:07:17.755540 2245 raft_consensus.cc:348] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:07:17.755882 2245 raft_consensus.cc:729] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b30fa096d74678a7e732e09451a96b, State: Initialized, Role: FOLLOWER
I20260620 23:07:17.755894 2189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:07:17.756116 2245 consensus_queue.cc:260] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:07:17.756394 2245 ts_tablet_manager.cc:1428] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260620 23:07:17.764125 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 2189
I20260620 23:07:17.764418 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:34783
--local_ip_for_outbound_sockets=127.17.188.66
--tserver_master_addrs=127.17.188.126:40003
--webserver_port=36491
--webserver_interface=127.17.188.66
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:17.836479 2357 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:07:17.756599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:43261)
0620 23:07:17.756737 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:17.756774 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:17.756894 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:17.836280 (+ 79386us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
W20260620 23:07:17.836761 2354 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:40003 (0 consecutive failures): Network error: Failed to ping master at 127.17.188.126:40003: Client connection negotiation failed: client connection to 127.17.188.126:40003: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:07:17.837132 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:17.837008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:39343)
0620 23:07:17.837072 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:17.837112 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 23:07:17.837486 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:17.837408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:36441)
0620 23:07:17.837450 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:17.837471 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 23:07:17.837599 2354 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:07:17.854425 2358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:17.854586 2358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:17.854609 2358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:17.855721 2358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:17.855754 2358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:17.855779 2358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:07:17.857101 2358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:34783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=36491
--tserver_master_addrs=127.17.188.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:17.857257 2358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:17.857489 2358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:17.862069 2366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:17.862139 2363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:17.862105 2364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:17.862152 2358 server_base.cc:1034] running on GCE node
I20260620 23:07:17.862486 2358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:17.862725 2358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:17.863915 2358 hybrid_clock.cc:648] HybridClock initialized: now 1781996837863903 us; error 35 us; skew 500 ppm
Jun 20 23:07:17 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996837, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:17.866276 2358 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:07:17.866696 2358 webserver.cc:460] Webserver started at http://127.17.188.66:36491/ using document root <none> and password file <none>
I20260620 23:07:17.866912 2358 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:17.866974 2358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:17.868183 2358 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:07:17.868721 2373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:17.868865 2358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:17.868959 2358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "9967ffca7fb3469ebd123c88ef441cde"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "8fbf4aadff7ea7a9eaa3397d2e16f19d"
server_key_iv: "778fe534dc013b1f0af872e309435468"
server_key_version: "encryptionkey@0"
I20260620 23:07:17.869273 2358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:17.884730 2358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:17.886445 2358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:17.886602 2358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:17.886832 2358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:17.887121 2358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:17.887171 2358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:17.887202 2358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:17.887240 2358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:17.893080 2358 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:34783
I20260620 23:07:17.893126 2486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:34783 every 8 connection(s)
I20260620 23:07:17.893502 2358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:07:17.894438 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:17.894117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:36179)
0620 23:07:17.894250 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:17.894314 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
W20260620 23:07:17.894881 2487 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:40003 (0 consecutive failures): Network error: Failed to ping master at 127.17.188.126:40003: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
I20260620 23:07:17.895915 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:17.895372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:40601)
0620 23:07:17.895862 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:17.895891 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472}
I20260620 23:07:17.896210 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:17.896128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:47389)
0620 23:07:17.896170 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:17.896192 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260620 23:07:17.896616 2487 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:07:17.899854 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 2358
I20260620 23:07:17.900107 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:33015
--local_ip_for_outbound_sockets=127.17.188.67
--tserver_master_addrs=127.17.188.126:40003
--webserver_port=33701
--webserver_interface=127.17.188.67
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:17.980723 2490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:17.980919 2490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:17.980954 2490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:17.982084 2490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:17.982146 2490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:17.982188 2490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:07:17.983618 2490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:33015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=33701
--tserver_master_addrs=127.17.188.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:17.983871 2490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:17.984088 2490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:17.988806 2496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:17.988796 2490 server_base.cc:1034] running on GCE node
W20260620 23:07:17.988782 2498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:17.988782 2495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:17.989161 2490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:17.989430 2490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:17.990583 2490 hybrid_clock.cc:648] HybridClock initialized: now 1781996837990574 us; error 35 us; skew 500 ppm
Jun 20 23:07:17 dist-test-slave-zpss krb5kdc[1051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996837, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:17.992869 2490 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:07:17.993245 2490 webserver.cc:460] Webserver started at http://127.17.188.67:33701/ using document root <none> and password file <none>
I20260620 23:07:17.993479 2490 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:17.993551 2490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:17.994736 2490 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:07:17.995256 2505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:17.995407 2490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:17.995492 2490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "16c5fa5d8bf04c4e95fd886479ceb85a"
format_stamp: "Formatted at 2026-06-20 23:06:36 on dist-test-slave-zpss"
server_key: "a0d5e0fee8ba2c1e6db7f5da154c4aab"
server_key_iv: "2bff0c579add74c2d914b7b9483afeec"
server_key_version: "encryptionkey@0"
I20260620 23:07:17.995791 2490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:18.006835 2490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:18.008471 2490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:18.008618 2490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:18.008818 2490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:18.009084 2490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:18.009135 2490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:18.009167 2490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:18.009194 2490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:18.014629 2490 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:33015
I20260620 23:07:18.014694 2618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:33015 every 8 connection(s)
I20260620 23:07:18.014976 2490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:07:18.015949 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:18.015582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:53649)
0620 23:07:18.015710 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:18.015794 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
W20260620 23:07:18.016297 2619 heartbeater.cc:644] Failed to heartbeat to 127.17.188.126:40003 (0 consecutive failures): Network error: Failed to ping master at 127.17.188.126:40003: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
I20260620 23:07:18.016525 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:18.016443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:38299)
0620 23:07:18.016482 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:18.016508 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 23:07:18.016746 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:18.016699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:36223)
0620 23:07:18.016718 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:18.016736 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260620 23:07:18.016837 2619 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:07:18.024514 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 2490
I20260620 23:07:18.838105 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:18.837879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:50479)
0620 23:07:18.838012 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:18.838047 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":57,"threads_started":1}
I20260620 23:07:18.897300 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:18.897105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:48127)
0620 23:07:18.897214 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:18.897242 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
I20260620 23:07:19.017297 2625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:19.017114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:56997)
0620 23:07:19.017223 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:19.017255 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260620 23:07:19.030148 2626 raft_consensus.cc:482] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:19.030265 2626 raft_consensus.cc:504] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:07:19.030627 2626 leader_election.cc:288] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783), 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015)
Jun 20 23:07:19 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996837, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.66@KRBTEST.COM
Jun 20 23:07:19 dist-test-slave-zpss krb5kdc[1051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996837, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.67@KRBTEST.COM
I20260620 23:07:19.041025 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:19.030905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:51911 (local address 127.17.188.67:33015)
0620 23:07:19.031027 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:19.031038 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:19.031070 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:07:19.031495 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:19.031497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:19.031545 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:19.031648 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:19.032387 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:19.033435 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:19.034388 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:19.034781 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:19.034895 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:19.037676 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:19.037679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:19.037680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:19.037688 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:19.039477 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:19.039820 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:19.039822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:19.039823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:19.039932 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:19.040111 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:19.040112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:19.040112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:19.040355 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:19.040441 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:19.040776 (+ 335us) server_negotiation.cc:299] Negotiation successful
0620 23:07:19.040902 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":52,"threads_started":1}
I20260620 23:07:19.041035 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:19.030871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:43725 (local address 127.17.188.66:34783)
0620 23:07:19.031077 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:19.031086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:19.031113 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:07:19.031563 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:19.031565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:19.031602 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:19.031751 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:19.032486 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:19.033416 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:19.034388 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:19.034781 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:19.034892 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:19.037275 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:19.037278 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:19.037279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:19.037287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:19.039262 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:19.039685 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:19.039688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:19.039688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:19.039795 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:19.040052 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:19.040053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:19.040054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:19.040355 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:19.040441 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:19.040776 (+ 335us) server_negotiation.cc:299] Negotiation successful
0620 23:07:19.040902 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":74,"threads_started":1}
I20260620 23:07:19.041033 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:19.030767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:33015 (local address 127.17.188.65:51911)
0620 23:07:19.030917 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:19.030927 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:19.030980 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:19.031709 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:19.031712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:19.032086 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:19.032275 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:19.032281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:19.033537 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:19.033537 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:19.034297 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:19.034299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:19.034877 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:19.034878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:19.034998 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:19.035559 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:19.035567 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:19.037570 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:19.039557 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:19.039558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:19.039558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:19.039744 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:19.039991 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:19.039992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:19.039992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:19.040049 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:19.040428 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:19.040436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:19.040583 (+ 147us) client_negotiation.cc:769] Sending connection context
0620 23:07:19.040712 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 23:07:19.040891 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":50,"thread_start_us":62,"threads_started":1}
I20260620 23:07:19.041095 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:19.030756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:34783 (local address 127.17.188.65:43725)
0620 23:07:19.030821 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:19.030831 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:19.030900 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:19.031747 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:19.031748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:19.032304 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:19.032388 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:19.032390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:19.033501 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:19.033502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:19.034298 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:19.034300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:19.034869 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:19.034869 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:19.034998 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:19.035624 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:19.035628 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:19.037181 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:19.039341 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:19.039347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:19.039348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:19.039586 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:19.039872 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:19.039872 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:19.039873 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:19.039985 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:19.040428 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:19.040436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:19.040583 (+ 147us) client_negotiation.cc:769] Sending connection context
0620 23:07:19.040712 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 23:07:19.040941 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":120}
W20260620 23:07:19.042083 2242 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
W20260620 23:07:19.042418 2240 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
I20260620 23:07:19.042461 2240 leader_election.cc:302] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6b30fa096d74678a7e732e09451a96b; no voters: 16c5fa5d8bf04c4e95fd886479ceb85a, 9967ffca7fb3469ebd123c88ef441cde
I20260620 23:07:19.042569 2626 raft_consensus.cc:2726] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:07:19.838721 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:19.838509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:35567)
0620 23:07:19.838637 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:19.838673 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260620 23:07:19.897990 2631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:19.897821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:47019)
0620 23:07:19.897922 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:19.897956 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260620 23:07:20.017848 2632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:20.017674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:39545)
0620 23:07:20.017783 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:20.017812 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260620 23:07:20.839378 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:20.839193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:39357)
0620 23:07:20.839302 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:20.839334 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260620 23:07:20.898511 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:20.898333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:45925)
0620 23:07:20.898446 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:20.898478 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260620 23:07:21.018406 2635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:21.018225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:60547)
0620 23:07:21.018342 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:21.018373 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260620 23:07:21.251880 2636 raft_consensus.cc:482] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:21.251972 2636 raft_consensus.cc:504] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:07:21.252175 2636 leader_election.cc:288] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783), 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015)
W20260620 23:07:21.252602 2240 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
W20260620 23:07:21.252671 2242 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
I20260620 23:07:21.252754 2240 leader_election.cc:302] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6b30fa096d74678a7e732e09451a96b; no voters: 16c5fa5d8bf04c4e95fd886479ceb85a, 9967ffca7fb3469ebd123c88ef441cde
I20260620 23:07:21.252840 2636 raft_consensus.cc:2726] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:07:21.840005 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:21.839790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:46323)
0620 23:07:21.839920 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:21.839951 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260620 23:07:21.899082 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:21.898875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:60981)
0620 23:07:21.899014 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:21.899048 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260620 23:07:22.018903 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:22.018711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:36515)
0620 23:07:22.018844 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:22.018870 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260620 23:07:22.840651 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:22.840439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:46201)
0620 23:07:22.840572 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:22.840605 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260620 23:07:22.899657 2641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:22.899469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:45739)
0620 23:07:22.899590 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:22.899622 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260620 23:07:23.019414 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:23.019244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:40667)
0620 23:07:23.019350 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:23.019380 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260620 23:07:23.841235 2643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:23.841040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:51147)
0620 23:07:23.841159 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:23.841191 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260620 23:07:23.900166 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:23.900004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:38229)
0620 23:07:23.900108 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:23.900135 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
I20260620 23:07:23.906414 2645 raft_consensus.cc:482] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:23.906497 2645 raft_consensus.cc:504] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:07:23.906677 2645 leader_election.cc:288] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783), 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015)
W20260620 23:07:23.906941 2240 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
W20260620 23:07:23.907076 2242 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
I20260620 23:07:23.907147 2242 leader_election.cc:302] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6b30fa096d74678a7e732e09451a96b; no voters: 16c5fa5d8bf04c4e95fd886479ceb85a, 9967ffca7fb3469ebd123c88ef441cde
I20260620 23:07:23.907235 2645 raft_consensus.cc:2726] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:07:24.019974 2646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:24.019791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:39219)
0620 23:07:24.019910 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:24.019941 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260620 23:07:24.841835 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:24.841637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:34459)
0620 23:07:24.841758 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:24.841792 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260620 23:07:24.900745 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:24.900563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:58693)
0620 23:07:24.900672 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:24.900704 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260620 23:07:25.020539 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:25.020370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:60875)
0620 23:07:25.020475 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:25.020507 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260620 23:07:25.842427 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:25.842229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:38035)
0620 23:07:25.842350 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:25.842383 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260620 23:07:25.901297 2651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:25.901126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:54175)
0620 23:07:25.901231 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:25.901264 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260620 23:07:26.021225 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:26.021002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:56831)
0620 23:07:26.021129 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:26.021180 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260620 23:07:26.843065 2653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:26.842836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:40057)
0620 23:07:26.842981 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:26.843019 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260620 23:07:26.902047 2654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:26.901843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:56259)
0620 23:07:26.901973 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:26.902008 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260620 23:07:27.021811 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:27.021605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:40311)
0620 23:07:27.021728 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:27.021759 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260620 23:07:27.254488 2656 raft_consensus.cc:482] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:27.254592 2656 raft_consensus.cc:504] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:07:27.254787 2656 leader_election.cc:288] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783), 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015)
W20260620 23:07:27.255168 2240 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
W20260620 23:07:27.255235 2242 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
I20260620 23:07:27.255324 2242 leader_election.cc:302] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6b30fa096d74678a7e732e09451a96b; no voters: 16c5fa5d8bf04c4e95fd886479ceb85a, 9967ffca7fb3469ebd123c88ef441cde
I20260620 23:07:27.255394 2656 raft_consensus.cc:2726] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:07:27.843819 2657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:27.843584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:40877)
0620 23:07:27.843726 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:27.843775 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260620 23:07:27.902640 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:27.902451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:40469)
0620 23:07:27.902557 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:27.902602 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260620 23:07:28.022377 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:28.022195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:51859)
0620 23:07:28.022298 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:28.022330 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
I20260620 23:07:28.844516 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:28.844297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:47875)
0620 23:07:28.844441 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:28.844474 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260620 23:07:28.903226 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:28.903048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:33529)
0620 23:07:28.903157 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:28.903189 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260620 23:07:29.022965 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:29.022759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:51367)
0620 23:07:29.022877 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:29.022931 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":54,"threads_started":1}
I20260620 23:07:29.247720 2663 raft_consensus.cc:482] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:29.247834 2663 raft_consensus.cc:504] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9967ffca7fb3469ebd123c88ef441cde" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34783 } } peers { permanent_uuid: "16c5fa5d8bf04c4e95fd886479ceb85a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 33015 } } peers { permanent_uuid: "b6b30fa096d74678a7e732e09451a96b" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 39691 } }
I20260620 23:07:29.248067 2663 leader_election.cc:288] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783), 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015)
W20260620 23:07:29.248440 2240 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9967ffca7fb3469ebd123c88ef441cde (127.17.188.66:34783): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
W20260620 23:07:29.248512 2242 leader_election.cc:341] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 16c5fa5d8bf04c4e95fd886479ceb85a (127.17.188.67:33015): Not found: Tablet not found: 19256433baf740b899aebff7393f9d38
I20260620 23:07:29.248580 2242 leader_election.cc:302] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6b30fa096d74678a7e732e09451a96b; no voters: 16c5fa5d8bf04c4e95fd886479ceb85a, 9967ffca7fb3469ebd123c88ef441cde
I20260620 23:07:29.248657 2663 raft_consensus.cc:2726] T 19256433baf740b899aebff7393f9d38 P b6b30fa096d74678a7e732e09451a96b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:07:29.845146 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:29.844950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:51237)
0620 23:07:29.845070 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:29.845102 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260620 23:07:29.903831 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:29.903645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:42591)
0620 23:07:29.903763 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:29.903791 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260620 23:07:30.023623 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:30.023418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:52231)
0620 23:07:30.023546 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:30.023582 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260620 23:07:30.845734 2667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:30.845523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:48699)
0620 23:07:30.845656 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:30.845690 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260620 23:07:30.904414 2668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:30.904228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:52889)
0620 23:07:30.904344 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:30.904377 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260620 23:07:31.024317 2669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:31.024030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:41791)
0620 23:07:31.024246 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:31.024281 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":50,"threads_started":1}
I20260620 23:07:31.846407 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:31.846217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:60285)
0620 23:07:31.846328 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:31.846365 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260620 23:07:31.905112 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:31.904869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:54569)
0620 23:07:31.905005 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:31.905061 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260620 23:07:32.024909 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:32.024723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:36211)
0620 23:07:32.024833 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:32.024867 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260620 23:07:32.847098 2673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:32.846918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.65:50497)
0620 23:07:32.847026 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:32.847056 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260620 23:07:32.905745 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:32.905559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.66:47215)
0620 23:07:32.905668 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:32.905703 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260620 23:07:33.025528 2675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:33.025342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:40003 (local address 127.17.188.67:50797)
0620 23:07:33.025460 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:33.025490 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.126:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":50,"threads_started":1}
I20260620 23:07:33.034204 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 2189
I20260620 23:07:33.038717 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 2358
I20260620 23:07:33.042789 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 2490
2026-06-20T23:07:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58198 ms)
[----------] 30 tests from SecurityITest (253959 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:33 dist-test-slave-zpss krb5kdc[2678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:33 dist-test-slave-zpss krb5kdc[2678](info): set up 2 sockets
Jun 20 23:07:33 dist-test-slave-zpss krb5kdc[2678](info): commencing operation
krb5kdc: starting...
W20260620 23:07:34.056982 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:34 dist-test-slave-zpss krb5kdc[2678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:07:34.165062 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:38167
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:41051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:38167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:34.242714 2691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:34.242959 2691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:34.243000 2691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:34.244134 2691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:34.244197 2691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:34.244223 2691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:34.244256 2691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:34.244303 2691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:34.245793 2691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:41051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:38167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:38167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:34.246052 2691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:34.246274 2691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:07:34.251202 2691 server_base.cc:1034] running on GCE node
W20260620 23:07:34.251155 2699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:34.251202 2697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:34.251173 2696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:34.251546 2691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:34.251857 2691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:34.253011 2691 hybrid_clock.cc:648] HybridClock initialized: now 1781996854252997 us; error 27 us; skew 500 ppm
Jun 20 23:07:34 dist-test-slave-zpss krb5kdc[2678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:34.255462 2691 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:07:34.255868 2691 webserver.cc:460] Webserver started at http://127.17.188.126:46879/ using document root <none> and password file <none>
I20260620 23:07:34.256049 2691 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:34.256086 2691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:34.256166 2691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:34.257062 2691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "5d877bf747e14022b1888e2e54d9cdb6"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "a12979f1157991b0979ca9e54d0594cb"
server_key_iv: "49f91f8bb2be17576436779e7829bf49"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.257422 2691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "5d877bf747e14022b1888e2e54d9cdb6"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "a12979f1157991b0979ca9e54d0594cb"
server_key_iv: "49f91f8bb2be17576436779e7829bf49"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.258661 2691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:07:34.259418 2706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.259562 2691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:34.259649 2691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "5d877bf747e14022b1888e2e54d9cdb6"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "a12979f1157991b0979ca9e54d0594cb"
server_key_iv: "49f91f8bb2be17576436779e7829bf49"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.259701 2691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:34.286298 2691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:34.288385 2691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:34.288481 2691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:34.292090 2691 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:38167
I20260620 23:07:34.292371 2758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:38167 every 8 connection(s)
I20260620 23:07:34.292445 2691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:34.293081 2759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:34.295218 2759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6: Bootstrap starting.
I20260620 23:07:34.295755 2759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:34.296034 2759 log.cc:824] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:34.296653 2759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6: No bootstrap required, opened a new log
I20260620 23:07:34.298161 2759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d877bf747e14022b1888e2e54d9cdb6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38167 } }
I20260620 23:07:34.298308 2759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:34.298350 2759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d877bf747e14022b1888e2e54d9cdb6, State: Initialized, Role: FOLLOWER
I20260620 23:07:34.298558 2759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d877bf747e14022b1888e2e54d9cdb6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38167 } }
I20260620 23:07:34.298655 2759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:34.298695 2759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:34.298744 2759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:34.299235 2759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d877bf747e14022b1888e2e54d9cdb6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38167 } }
I20260620 23:07:34.299350 2759 leader_election.cc:302] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d877bf747e14022b1888e2e54d9cdb6; no voters:
I20260620 23:07:34.299413 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 2691
I20260620 23:07:34.299530 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:34.299539 2759 leader_election.cc:288] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:34.299561 2762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:34.299722 2762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [term 1 LEADER]: Becoming Leader. State: Replica: 5d877bf747e14022b1888e2e54d9cdb6, State: Running, Role: LEADER
I20260620 23:07:34.299734 18161 external_mini_cluster.cc:1428] Setting key 8b0353db3f53bb9abdb683cf672fbee1
I20260620 23:07:34.299787 2759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:34.299883 2762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d877bf747e14022b1888e2e54d9cdb6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38167 } }
I20260620 23:07:34.300431 2763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d877bf747e14022b1888e2e54d9cdb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d877bf747e14022b1888e2e54d9cdb6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38167 } } }
I20260620 23:07:34.300531 2763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:34.300441 2764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d877bf747e14022b1888e2e54d9cdb6. Latest consensus state: current_term: 1 leader_uuid: "5d877bf747e14022b1888e2e54d9cdb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d877bf747e14022b1888e2e54d9cdb6" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38167 } } }
I20260620 23:07:34.300722 2764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:34.301074 2773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:34.301859 2773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:34.303990 2773 catalog_manager.cc:1347] Generated new cluster ID: b445060a3b73486baf82572f5ae92106
I20260620 23:07:34.304051 2773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:07:34 dist-test-slave-zpss krb5kdc[2678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:34.310722 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.300216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.17.188.126:38167)
0620 23:07:34.300376 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.300384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.300418 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:07:34.300535 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.300541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.300595 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.301444 (+ 849us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:34.301668 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.302904 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.303654 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.304297 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.304397 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.307109 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:34.307112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:34.307113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:34.307123 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:34.308939 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.309411 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.309414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.309414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.309517 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.309786 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.309787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.309787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.310053 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:34.310185 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.310322 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.310438 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260620 23:07:34.342598 2773 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:34.343124 2773 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:34.348459 2773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6: Generated new TSK 0
I20260620 23:07:34.348621 2773 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:07:34.378074 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38167
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:41051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:34.454617 2785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:34.454794 2785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:34.454814 2785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:34.455897 2785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:34.455935 2785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:34.455981 2785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:07:34.457392 2785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:41051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:34.457602 2785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:34.457804 2785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:34.462476 2791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:34.462646 2790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:34.462749 2793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:34.462942 2785 server_base.cc:1034] running on GCE node
I20260620 23:07:34.463210 2785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:34.463455 2785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:34.464628 2785 hybrid_clock.cc:648] HybridClock initialized: now 1781996854464616 us; error 39 us; skew 500 ppm
Jun 20 23:07:34 dist-test-slave-zpss krb5kdc[2678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:34.467101 2785 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:07:34.467502 2785 webserver.cc:460] Webserver started at http://127.17.188.65:46707/ using document root <none> and password file <none>
I20260620 23:07:34.467720 2785 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:34.467783 2785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:34.467926 2785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:34.468854 2785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "d2e5629f78a6414b9f9846a5e746f685"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "7292c0b13d97624773f7255eb40401df"
server_key_iv: "74414b6ca495cceae0952556c0113af7"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.469259 2785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "d2e5629f78a6414b9f9846a5e746f685"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "7292c0b13d97624773f7255eb40401df"
server_key_iv: "74414b6ca495cceae0952556c0113af7"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.470512 2785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:07:34.471207 2800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.471489 2785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:07:34.471598 2785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "d2e5629f78a6414b9f9846a5e746f685"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "7292c0b13d97624773f7255eb40401df"
server_key_iv: "74414b6ca495cceae0952556c0113af7"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.471684 2785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:34.477809 2785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:34.479962 2785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:34.480109 2785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:34.480324 2785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:34.480602 2785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:34.480654 2785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.480691 2785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:34.480719 2785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.486343 2785 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:36959
I20260620 23:07:34.486379 2913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:36959 every 8 connection(s)
I20260620 23:07:34.486665 2785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:07:34.492110 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 2785
I20260620 23:07:34.492218 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:07:34.492385 18161 external_mini_cluster.cc:1428] Setting key 58b8ea9b17bd486d59dd0f749e2e2bf5
Jun 20 23:07:34 dist-test-slave-zpss krb5kdc[2678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:34.495828 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.487317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:41805 (local address 127.17.188.126:38167)
0620 23:07:34.487377 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.487379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.487615 (+ 236us) server_negotiation.cc:407] Connection header received
0620 23:07:34.488076 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.488077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.488089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.488156 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:34.489047 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.489624 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.490579 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.490780 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.490879 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.493116 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:34.493118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:34.493118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:34.493126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:34.494453 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.494866 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.494868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.494869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.494929 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.495180 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.495181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.495181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.495304 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:34.495377 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.495642 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.495728 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:07:34.495981 2916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.487295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:38167 (local address 127.17.188.65:41805)
0620 23:07:34.487457 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:34.487507 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:34.487975 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:34.488221 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:34.488224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:34.488786 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:34.488951 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.488956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.489725 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.489726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.490485 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.490487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.490890 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.490891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.491007 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.491498 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:34.491505 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:34.493026 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:34.494553 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:34.494560 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:34.494562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:34.494774 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:34.495002 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:34.495003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:34.495003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:34.495112 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:34.495380 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:34.495387 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:34.495515 (+ 128us) client_negotiation.cc:769] Sending connection context
0620 23:07:34.495620 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 23:07:34.495765 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260620 23:07:34.496446 2914 heartbeater.cc:347] Connected to a master server at 127.17.188.126:38167
I20260620 23:07:34.496546 2914 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:34.496726 2914 heartbeater.cc:511] Master 127.17.188.126:38167 requested a full tablet report, sending...
I20260620 23:07:34.497264 2723 ts_manager.cc:194] Registered new tserver with Master: d2e5629f78a6414b9f9846a5e746f685 (127.17.188.65:36959)
I20260620 23:07:34.498270 2723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:41805
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:07:34.515684 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38167
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:41051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:34.592273 2919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:34.592463 2919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:34.592482 2919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:34.593616 2919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:34.593654 2919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:34.593706 2919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:07:34.595060 2919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:41051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:34.595286 2919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:34.595510 2919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:34.600294 2927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:34.600342 2924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:34.600296 2925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:34.600368 2919 server_base.cc:1034] running on GCE node
I20260620 23:07:34.600726 2919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:34.600935 2919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:34.602082 2919 hybrid_clock.cc:648] HybridClock initialized: now 1781996854602063 us; error 33 us; skew 500 ppm
Jun 20 23:07:34 dist-test-slave-zpss krb5kdc[2678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:34.604485 2919 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:07:34.604871 2919 webserver.cc:460] Webserver started at http://127.17.188.66:38733/ using document root <none> and password file <none>
I20260620 23:07:34.605083 2919 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:34.605149 2919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:34.605291 2919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:34.606228 2919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "99901803e90f449da3098809e3edd7e1"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "7b93a90789a1c12000b59e38f29e4741"
server_key_iv: "19360a95389b225082c353e3762b40b7"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.606562 2919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "99901803e90f449da3098809e3edd7e1"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "7b93a90789a1c12000b59e38f29e4741"
server_key_iv: "19360a95389b225082c353e3762b40b7"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.607923 2919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:07:34.608582 2934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.608754 2919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:34.608846 2919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "99901803e90f449da3098809e3edd7e1"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "7b93a90789a1c12000b59e38f29e4741"
server_key_iv: "19360a95389b225082c353e3762b40b7"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.608934 2919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:34.642338 2919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:34.644575 2919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:34.644750 2919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:34.644966 2919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:34.645290 2919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:34.645361 2919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.645402 2919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:34.645432 2919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.651100 2919 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:32943
I20260620 23:07:34.651144 3047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:32943 every 8 connection(s)
I20260620 23:07:34.651434 2919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:07:34 dist-test-slave-zpss krb5kdc[2678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:34.660179 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.652097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:33579 (local address 127.17.188.126:38167)
0620 23:07:34.652170 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.652171 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.652363 (+ 192us) server_negotiation.cc:407] Connection header received
0620 23:07:34.652855 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.652856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.652867 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.652920 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:34.653875 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.654388 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.655347 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.655585 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.655674 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.657660 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:34.657662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:34.657662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:34.657670 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:34.658823 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.659239 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.659240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.659241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.659300 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.659543 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.659544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.659545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.659663 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:34.659741 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.660005 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.660089 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 23:07:34.660243 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 2919
I20260620 23:07:34.660357 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:07:34.660372 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.652088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:38167 (local address 127.17.188.66:33579)
0620 23:07:34.652254 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:34.652285 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:34.652753 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:34.652999 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:34.653002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:34.653600 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:34.653781 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.653785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.654491 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.654493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.655258 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.655260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.655683 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.655684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.655800 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.656265 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:34.656272 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:34.657566 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:34.658917 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:34.658923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:34.658925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:34.659146 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:34.659371 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:34.659371 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:34.659372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:34.659477 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:34.659730 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:34.659736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:34.659878 (+ 142us) client_negotiation.cc:769] Sending connection context
0620 23:07:34.659983 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 23:07:34.660143 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":91,"threads_started":1}
I20260620 23:07:34.660552 18161 external_mini_cluster.cc:1428] Setting key 51b9832da38beb0a2a9fb412d8b46d6b
I20260620 23:07:34.660774 3048 heartbeater.cc:347] Connected to a master server at 127.17.188.126:38167
I20260620 23:07:34.660857 3048 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:34.661047 3048 heartbeater.cc:511] Master 127.17.188.126:38167 requested a full tablet report, sending...
I20260620 23:07:34.661453 2723 ts_manager.cc:194] Registered new tserver with Master: 99901803e90f449da3098809e3edd7e1 (127.17.188.66:32943)
I20260620 23:07:34.661967 2723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:33579
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:07:34.683682 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38167
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:41051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:34.760891 3053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:34.761103 3053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:34.761142 3053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:34.762384 3053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:34.762444 3053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:34.762486 3053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:07:34.763988 3053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:41051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:34.764272 3053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:34.764503 3053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:34.769296 3061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:34.769304 3058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:34.769294 3059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:34.769518 3053 server_base.cc:1034] running on GCE node
I20260620 23:07:34.769730 3053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:34.769979 3053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:34.771118 3053 hybrid_clock.cc:648] HybridClock initialized: now 1781996854771109 us; error 23 us; skew 500 ppm
Jun 20 23:07:34 dist-test-slave-zpss krb5kdc[2678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:34.773603 3053 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:07:34.773969 3053 webserver.cc:460] Webserver started at http://127.17.188.67:44861/ using document root <none> and password file <none>
I20260620 23:07:34.774152 3053 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:34.774191 3053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:34.774354 3053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:34.775333 3053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "ae6e51f46e014fb8b0fadf67564caa71"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "4ff69742c1b22ff913ea14634c557186"
server_key_iv: "9577155a5d9cd110560540654435c0ce"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.775715 3053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "ae6e51f46e014fb8b0fadf67564caa71"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "4ff69742c1b22ff913ea14634c557186"
server_key_iv: "9577155a5d9cd110560540654435c0ce"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.776947 3053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:07:34.777732 3068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.777899 3053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.777993 3053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "ae6e51f46e014fb8b0fadf67564caa71"
format_stamp: "Formatted at 2026-06-20 23:07:34 on dist-test-slave-zpss"
server_key: "4ff69742c1b22ff913ea14634c557186"
server_key_iv: "9577155a5d9cd110560540654435c0ce"
server_key_version: "encryptionkey@0"
I20260620 23:07:34.778070 3053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:34.808943 3053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:34.811308 3053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:34.811524 3053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:34.811738 3053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:34.812024 3053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:34.812073 3053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.812112 3053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:34.812136 3053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:34.817828 3053 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:42277
I20260620 23:07:34.817901 3181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:42277 every 8 connection(s)
I20260620 23:07:34.818212 3053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:07:34 dist-test-slave-zpss krb5kdc[2678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:34.826537 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.818938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:52953 (local address 127.17.188.126:38167)
0620 23:07:34.818983 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.818986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.819185 (+ 199us) server_negotiation.cc:407] Connection header received
0620 23:07:34.819628 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.819628 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.819638 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.819685 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:34.820535 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.821012 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.821979 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.822154 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.822205 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.824089 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:34.824091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:34.824091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:34.824097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:34.825500 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.825827 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.825828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.825829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.825875 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.826066 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.826067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.826067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.826155 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:34.826203 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.826423 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.826476 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:07:34.826682 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.818925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:38167 (local address 127.17.188.67:52953)
0620 23:07:34.819068 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:34.819101 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:34.819547 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:34.819747 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:34.819750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:34.820288 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:34.820465 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.820469 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.821084 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.821084 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.821905 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.821907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.822223 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.822224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.822321 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.822701 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:34.822709 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:34.824010 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:34.825563 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:34.825568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:34.825569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:34.825760 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:34.825929 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:34.825929 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:34.825930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:34.826016 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:34.826219 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:34.826222 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:34.826319 (+ 97us) client_negotiation.cc:769] Sending connection context
0620 23:07:34.826395 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:07:34.826501 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260620 23:07:34.827144 3182 heartbeater.cc:347] Connected to a master server at 127.17.188.126:38167
I20260620 23:07:34.827242 3182 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:34.827416 3182 heartbeater.cc:511] Master 127.17.188.126:38167 requested a full tablet report, sending...
I20260620 23:07:34.827741 2723 ts_manager.cc:194] Registered new tserver with Master: ae6e51f46e014fb8b0fadf67564caa71 (127.17.188.67:42277)
I20260620 23:07:34.827970 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 3053
I20260620 23:07:34.828065 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:07:34.828148 2723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:52953
I20260620 23:07:34.828248 18161 external_mini_cluster.cc:1428] Setting key 65dcbd68eb9805d339c03e49667f5bac
I20260620 23:07:34.828971 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:07:34.835817 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.830262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.17.188.126:38167)
0620 23:07:34.830306 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.830309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.830367 (+ 58us) server_negotiation.cc:407] Connection header received
0620 23:07:34.830441 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.830442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.830450 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.830521 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:34.831049 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.831477 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.832199 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.832416 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.832486 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.833610 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:34.833612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:34.833612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:34.833618 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:34.834687 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.835038 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.835039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.835039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.835094 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:34.835314 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:34.835315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:34.835316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:34.835443 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:34.835541 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.835645 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.835717 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:07:34.837141 2723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:07:34.838696 2723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:07:34.844729 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.840855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:32943 (local address 127.0.0.1:51526)
0620 23:07:34.841431 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:34.841442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:34.841490 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:34.841777 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:34.841778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:34.841780 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:34.841820 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.841822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.842922 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.842923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.843805 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.843807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.844461 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.844462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.844537 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.844549 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:07:34.844616 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 23:07:34.844665 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":50,"threads_started":1}
I20260620 23:07:34.844748 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.840945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.17.188.66:32943)
0620 23:07:34.841089 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.841094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.841556 (+ 462us) server_negotiation.cc:407] Connection header received
0620 23:07:34.841602 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.841616 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.841631 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.841698 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:34.841901 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.842797 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.843886 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.844364 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.844457 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.844512 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.844635 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.844674 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260620 23:07:34.845121 3190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.840643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36959 (local address 127.0.0.1:53706)
0620 23:07:34.840976 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:34.841003 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:34.841065 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:34.841300 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:34.841307 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:34.841309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:34.841553 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.841556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.842491 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.842492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.843855 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.843858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.844797 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.844798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.844879 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.844885 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:07:34.844934 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:07:34.844982 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":64,"thread_start_us":56,"threads_started":1}
I20260620 23:07:34.845137 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.840823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.17.188.65:36959)
0620 23:07:34.841138 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.841141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.841150 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:07:34.841192 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.841194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.841206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.841301 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:34.841646 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.842393 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.843957 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.844647 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.844713 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.844761 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.845022 (+ 261us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.845064 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":77,"threads_started":1}
I20260620 23:07:34.845160 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.840719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.17.188.67:42277)
0620 23:07:34.840967 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.840972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.841143 (+ 171us) server_negotiation.cc:407] Connection header received
0620 23:07:34.841179 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.841181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.841196 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.841287 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:34.841471 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.842455 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.843954 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.844646 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.844712 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.844760 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.845058 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.845090 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":98,"threads_started":1}
I20260620 23:07:34.845280 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.840643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:42277 (local address 127.0.0.1:51218)
0620 23:07:34.840965 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:34.841003 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:34.841065 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:34.841298 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:34.841302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:34.841304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:34.841408 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.841411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.842546 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.842546 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.843861 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.843863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.844796 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.844798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.844879 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.844886 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 23:07:34.844934 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 23:07:34.844981 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":94,"thread_start_us":57,"threads_started":1}
I20260620 23:07:34.845999 2848 tablet_service.cc:1467] Processing CreateTablet for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b (DEFAULT_TABLE table=test-table [id=b7eb8aecf81548a38d974511d087b757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:34.846292 2848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:34.848822 3196 tablet_bootstrap.cc:492] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685: Bootstrap starting.
I20260620 23:07:34.849152 3116 tablet_service.cc:1467] Processing CreateTablet for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b (DEFAULT_TABLE table=test-table [id=b7eb8aecf81548a38d974511d087b757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:34.849457 3196 tablet_bootstrap.cc:654] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:34.849473 3116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:34.850077 3196 log.cc:824] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:34.850802 3196 tablet_bootstrap.cc:492] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685: No bootstrap required, opened a new log
I20260620 23:07:34.850888 3196 ts_tablet_manager.cc:1397] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:07:34.852640 3198 tablet_bootstrap.cc:492] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: Bootstrap starting.
I20260620 23:07:34.852967 3196 raft_consensus.cc:348] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } }
I20260620 23:07:34.853148 3196 raft_consensus.cc:374] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:34.853188 3196 raft_consensus.cc:729] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e5629f78a6414b9f9846a5e746f685, State: Initialized, Role: FOLLOWER
I20260620 23:07:34.853199 3198 tablet_bootstrap.cc:654] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:34.853220 2982 tablet_service.cc:1467] Processing CreateTablet for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b (DEFAULT_TABLE table=test-table [id=b7eb8aecf81548a38d974511d087b757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:34.853298 3196 consensus_queue.cc:260] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } }
I20260620 23:07:34.853535 2982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:34.853580 3196 ts_tablet_manager.cc:1428] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:07:34.853799 3198 log.cc:824] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:34.854449 3198 tablet_bootstrap.cc:492] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: No bootstrap required, opened a new log
I20260620 23:07:34.854528 3198 ts_tablet_manager.cc:1397] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:34.855785 3198 raft_consensus.cc:348] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } }
I20260620 23:07:34.855856 3201 tablet_bootstrap.cc:492] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: Bootstrap starting.
I20260620 23:07:34.855903 2914 heartbeater.cc:503] Master 127.17.188.126:38167 was elected leader, sending a full tablet report...
I20260620 23:07:34.855944 3198 raft_consensus.cc:374] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:34.856003 3198 raft_consensus.cc:729] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae6e51f46e014fb8b0fadf67564caa71, State: Initialized, Role: FOLLOWER
I20260620 23:07:34.856526 3201 tablet_bootstrap.cc:654] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:34.856204 3198 consensus_queue.cc:260] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } }
I20260620 23:07:34.856868 3182 heartbeater.cc:503] Master 127.17.188.126:38167 was elected leader, sending a full tablet report...
I20260620 23:07:34.856849 3198 ts_tablet_manager.cc:1428] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:07:34.857427 3201 log.cc:824] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:34.858060 3201 tablet_bootstrap.cc:492] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: No bootstrap required, opened a new log
I20260620 23:07:34.858136 3201 ts_tablet_manager.cc:1397] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:07:34.859390 3201 raft_consensus.cc:348] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } }
I20260620 23:07:34.859547 3201 raft_consensus.cc:374] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:34.859591 3201 raft_consensus.cc:729] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99901803e90f449da3098809e3edd7e1, State: Initialized, Role: FOLLOWER
I20260620 23:07:34.859705 3201 consensus_queue.cc:260] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } }
I20260620 23:07:34.859946 3201 ts_tablet_manager.cc:1428] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:07:34.859987 3048 heartbeater.cc:503] Master 127.17.188.126:38167 was elected leader, sending a full tablet report...
W20260620 23:07:34.901954 3049 tablet.cc:2369] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:07:34.935218 3204 raft_consensus.cc:482] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:34.935341 3204 raft_consensus.cc:504] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } }
I20260620 23:07:34.935734 3204 leader_election.cc:288] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae6e51f46e014fb8b0fadf67564caa71 (127.17.188.67:42277), d2e5629f78a6414b9f9846a5e746f685 (127.17.188.65:36959)
I20260620 23:07:34.938575 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.935992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:47611 (local address 127.17.188.67:42277)
0620 23:07:34.936033 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.936035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.936045 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:07:34.936099 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.936100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.936110 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.936187 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:34.936341 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.936829 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.937847 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.938216 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.938319 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.938328 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.938473 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.938511 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:07:34.938575 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.935902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:42277 (local address 127.17.188.66:47611)
0620 23:07:34.935985 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:34.935997 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:34.936041 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:34.936194 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:34.936195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:34.936197 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:34.936271 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.936273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.936917 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.936918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.937750 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.937752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.938322 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.938323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.938399 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.938402 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:34.938468 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 23:07:34.938506 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260620 23:07:34.938685 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.935984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:37005 (local address 127.17.188.65:36959)
0620 23:07:34.936032 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.936035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.936222 (+ 187us) server_negotiation.cc:407] Connection header received
0620 23:07:34.936273 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.936274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.936287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.936378 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:34.936546 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.936992 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.937945 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.938320 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.938399 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.938407 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.938565 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.938606 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:07:34.938882 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.935908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36959 (local address 127.17.188.66:37005)
0620 23:07:34.936153 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:34.936161 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:34.936223 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:34.936382 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:34.936383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:34.936384 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:34.936476 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.936477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.937078 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.937079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.937854 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:34.937856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.938413 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.938414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:34.938484 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.938487 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:34.938545 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 23:07:34.938583 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":62,"thread_start_us":111,"threads_started":1}
I20260620 23:07:34.939013 3136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b8cf9f7dd847f8ae7c0adfb8cc834b" candidate_uuid: "99901803e90f449da3098809e3edd7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae6e51f46e014fb8b0fadf67564caa71" is_pre_election: true
I20260620 23:07:34.939160 3136 raft_consensus.cc:2454] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99901803e90f449da3098809e3edd7e1 in term 0.
I20260620 23:07:34.939234 2868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b8cf9f7dd847f8ae7c0adfb8cc834b" candidate_uuid: "99901803e90f449da3098809e3edd7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e5629f78a6414b9f9846a5e746f685" is_pre_election: true
I20260620 23:07:34.939363 2868 raft_consensus.cc:2454] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99901803e90f449da3098809e3edd7e1 in term 0.
I20260620 23:07:34.939376 2936 leader_election.cc:302] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99901803e90f449da3098809e3edd7e1, ae6e51f46e014fb8b0fadf67564caa71; no voters:
I20260620 23:07:34.939543 3204 raft_consensus.cc:2781] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:07:34.939596 3204 raft_consensus.cc:482] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:07:34.939617 3204 raft_consensus.cc:3037] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:34.940332 3204 raft_consensus.cc:504] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } }
I20260620 23:07:34.940469 3204 leader_election.cc:288] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [CANDIDATE]: Term 1 election: Requested vote from peers ae6e51f46e014fb8b0fadf67564caa71 (127.17.188.67:42277), d2e5629f78a6414b9f9846a5e746f685 (127.17.188.65:36959)
I20260620 23:07:34.940639 3136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b8cf9f7dd847f8ae7c0adfb8cc834b" candidate_uuid: "99901803e90f449da3098809e3edd7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae6e51f46e014fb8b0fadf67564caa71"
I20260620 23:07:34.940649 2868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b8cf9f7dd847f8ae7c0adfb8cc834b" candidate_uuid: "99901803e90f449da3098809e3edd7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e5629f78a6414b9f9846a5e746f685"
I20260620 23:07:34.940719 3136 raft_consensus.cc:3037] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:34.940721 2868 raft_consensus.cc:3037] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:34.941619 2868 raft_consensus.cc:2454] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99901803e90f449da3098809e3edd7e1 in term 1.
I20260620 23:07:34.941619 3136 raft_consensus.cc:2454] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99901803e90f449da3098809e3edd7e1 in term 1.
I20260620 23:07:34.941784 2936 leader_election.cc:302] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99901803e90f449da3098809e3edd7e1, ae6e51f46e014fb8b0fadf67564caa71; no voters:
I20260620 23:07:34.941874 3204 raft_consensus.cc:2781] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:34.942019 3204 raft_consensus.cc:686] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 1 LEADER]: Becoming Leader. State: Replica: 99901803e90f449da3098809e3edd7e1, State: Running, Role: LEADER
I20260620 23:07:34.942140 3204 consensus_queue.cc:237] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } }
I20260620 23:07:34.942943 2722 catalog_manager.cc:5495] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99901803e90f449da3098809e3edd7e1 (127.17.188.66). New cstate: current_term: 1 leader_uuid: "99901803e90f449da3098809e3edd7e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99901803e90f449da3098809e3edd7e1" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32943 } health_report { overall_health: HEALTHY } } }
I20260620 23:07:34.959245 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.956969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.17.188.66:32943)
0620 23:07:34.957025 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:34.957028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:34.957036 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:07:34.957105 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:34.957106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:34.957123 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:34.957203 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:07:34.957347 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.957779 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.958520 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:34.958697 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:34.958774 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:34.958940 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:07:34.959035 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:07:34.959119 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:34.959136 (+ 17us) server_negotiation.cc:299] Negotiation successful
0620 23:07:34.959154 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:07:34.960879 2868 raft_consensus.cc:1261] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P d2e5629f78a6414b9f9846a5e746f685 [term 1 FOLLOWER]: Refusing update from remote peer 99901803e90f449da3098809e3edd7e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:34.960879 3136 raft_consensus.cc:1261] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [term 1 FOLLOWER]: Refusing update from remote peer 99901803e90f449da3098809e3edd7e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:34.961222 3204 consensus_queue.cc:1035] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae6e51f46e014fb8b0fadf67564caa71" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 42277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:34.961488 3206 consensus_queue.cc:1035] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e5629f78a6414b9f9846a5e746f685" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 36959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:34.963697 3209 mvcc.cc:205] Tried to move back new op lower bound from 7299059117917249536 to 7299059117843664896. Current Snapshot: MvccSnapshot[applied={T|T < 7299059117917249536}]
I20260620 23:07:34.964447 3211 mvcc.cc:205] Tried to move back new op lower bound from 7299059117917249536 to 7299059117843664896. Current Snapshot: MvccSnapshot[applied={T|T < 7299059117917249536}]
I20260620 23:07:34.965889 2723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:34.965992 2723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:34.966665 2723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d877bf747e14022b1888e2e54d9cdb6: Sending DeleteTablet for 3 replicas of tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b
I20260620 23:07:34.966998 2848 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:34 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:53706
I20260620 23:07:34.967015 3116 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:34 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:51218
I20260620 23:07:34.967216 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 2785
I20260620 23:07:34.967787 2982 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:34 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:51526
I20260620 23:07:34.968036 3220 tablet_replica.cc:330] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: stopping tablet replica
I20260620 23:07:34.968152 3220 raft_consensus.cc:2229] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:07:34.968263 3220 raft_consensus.cc:2258] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:34.968446 3219 tablet_replica.cc:330] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: stopping tablet replica
I20260620 23:07:34.968534 3219 raft_consensus.cc:2229] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:34.968626 3219 raft_consensus.cc:2258] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:34.968974 3220 ts_tablet_manager.cc:1905] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:34.970431 3220 ts_tablet_manager.cc:1918] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:34.970506 3220 log.cc:1196] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/f2b8cf9f7dd847f8ae7c0adfb8cc834b
I20260620 23:07:34.970616 3219 ts_tablet_manager.cc:1905] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:34.970789 3220 ts_tablet_manager.cc:1939] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P 99901803e90f449da3098809e3edd7e1: Deleting consensus metadata
I20260620 23:07:34.971153 2708 catalog_manager.cc:4841] TS 99901803e90f449da3098809e3edd7e1 (127.17.188.66:32943): tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b (table test-table [id=b7eb8aecf81548a38d974511d087b757]) successfully deleted
I20260620 23:07:34.972123 3219 ts_tablet_manager.cc:1918] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:34.972222 3219 log.cc:1196] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/f2b8cf9f7dd847f8ae7c0adfb8cc834b
I20260620 23:07:34.972458 3219 ts_tablet_manager.cc:1939] T f2b8cf9f7dd847f8ae7c0adfb8cc834b P ae6e51f46e014fb8b0fadf67564caa71: Deleting consensus metadata
I20260620 23:07:34.972815 2708 catalog_manager.cc:4841] TS ae6e51f46e014fb8b0fadf67564caa71 (127.17.188.67:42277): tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b (table test-table [id=b7eb8aecf81548a38d974511d087b757]) successfully deleted
W20260620 23:07:34.974783 2709 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:36959 (error 108)
I20260620 23:07:34.975036 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 2919
I20260620 23:07:34.975848 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:34.975112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:36959 (local address 127.0.0.1:53722)
0620 23:07:34.975779 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:34.975823 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":648}
W20260620 23:07:34.976001 2709 catalog_manager.cc:4584] TS d2e5629f78a6414b9f9846a5e746f685 (127.17.188.65:36959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2b8cf9f7dd847f8ae7c0adfb8cc834b: Network error: Client connection negotiation failed: client connection to 127.17.188.65:36959: connect: Connection refused (error 111)
I20260620 23:07:34.979671 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 3053
I20260620 23:07:34.983949 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 2691
2026-06-20T23:07:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1941 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T23:07:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:35Z Disabled control of system clock
I20260620 23:07:35.010298 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42809
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:40049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:07:35.089570 3225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:35.089790 3225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:35.089811 3225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:35.090919 3225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:35.090955 3225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:35.090967 3225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:35.090981 3225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:35.090993 3225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:35.092242 3225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:40049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:35.092437 3225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:35.092670 3225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:35.097397 3231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:35.097481 3225 server_base.cc:1034] running on GCE node
W20260620 23:07:35.097394 3233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:35.097394 3230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:35.097911 3225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:35.098215 3225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:35.099392 3225 hybrid_clock.cc:648] HybridClock initialized: now 1781996855099389 us; error 37 us; skew 500 ppm
I20260620 23:07:35.100541 3225 webserver.cc:460] Webserver started at http://127.17.188.126:42887/ using document root <none> and password file <none>
I20260620 23:07:35.100775 3225 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:35.100844 3225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:35.100978 3225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:35.102136 3225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "6b9e5334e2174203849f39a7259f4f4e"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "d91f28d6433706e73bdb18eb27ec6ef8"
server_key_iv: "d749d919a5369d13a272f64b18024d18"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.102499 3225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "6b9e5334e2174203849f39a7259f4f4e"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "d91f28d6433706e73bdb18eb27ec6ef8"
server_key_iv: "d749d919a5369d13a272f64b18024d18"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.103767 3225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:07:35.104467 3239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.104628 3225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:35.104724 3225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "6b9e5334e2174203849f39a7259f4f4e"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "d91f28d6433706e73bdb18eb27ec6ef8"
server_key_iv: "d749d919a5369d13a272f64b18024d18"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.104810 3225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:35.122226 3225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:35.122503 3225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:35.122646 3225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:35.126421 3225 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:42809
I20260620 23:07:35.126452 3291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:42809 every 8 connection(s)
I20260620 23:07:35.126753 3225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:35.127599 3292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:35.129686 3292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e: Bootstrap starting.
I20260620 23:07:35.130198 3292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:35.130426 3292 log.cc:824] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:35.131004 3292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e: No bootstrap required, opened a new log
I20260620 23:07:35.132486 3292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9e5334e2174203849f39a7259f4f4e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42809 } }
I20260620 23:07:35.132615 3292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:35.132637 3292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9e5334e2174203849f39a7259f4f4e, State: Initialized, Role: FOLLOWER
I20260620 23:07:35.132764 3292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9e5334e2174203849f39a7259f4f4e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42809 } }
I20260620 23:07:35.132858 3292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:35.132881 3292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:35.132915 3292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:35.133370 3292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9e5334e2174203849f39a7259f4f4e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42809 } }
I20260620 23:07:35.133447 3292 leader_election.cc:302] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b9e5334e2174203849f39a7259f4f4e; no voters:
I20260620 23:07:35.133558 3292 leader_election.cc:288] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:35.133630 3295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:35.133809 3295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [term 1 LEADER]: Becoming Leader. State: Replica: 6b9e5334e2174203849f39a7259f4f4e, State: Running, Role: LEADER
I20260620 23:07:35.133838 3292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:35.133906 3295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9e5334e2174203849f39a7259f4f4e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42809 } }
I20260620 23:07:35.134187 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 3225
I20260620 23:07:35.134289 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:35.134532 18161 external_mini_cluster.cc:1428] Setting key f33502fc691d2ccd11f132c10dc644d2
I20260620 23:07:35.135047 3297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b9e5334e2174203849f39a7259f4f4e. Latest consensus state: current_term: 1 leader_uuid: "6b9e5334e2174203849f39a7259f4f4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9e5334e2174203849f39a7259f4f4e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42809 } } }
I20260620 23:07:35.135161 3297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:35.135362 3296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b9e5334e2174203849f39a7259f4f4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9e5334e2174203849f39a7259f4f4e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42809 } } }
I20260620 23:07:35.135444 3311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:35.135577 3296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:35.136019 3311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:35.137753 3311 catalog_manager.cc:1347] Generated new cluster ID: 7b227a83523a4ff683828d95dcc8d286
I20260620 23:07:35.137813 3311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:35.140565 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.135195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.17.188.126:42809)
0620 23:07:35.135552 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.135561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.135592 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:07:35.135703 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.135710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.135772 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.135924 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:35.136188 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.137397 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.138484 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.139089 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.139211 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.139866 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:35.139870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:35.139870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:35.140088 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:35.140129 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:35.140144 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:35.140216 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.140284 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.140361 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":82,"threads_started":1}
I20260620 23:07:35.146206 3311 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:35.146893 3311 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:35.149420 3311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e: Generated new TSK 0
I20260620 23:07:35.149567 3311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:35.154402 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42809
--builtin_ntp_servers=127.17.188.84:40049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:07:35.262576 3316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:35.262763 3316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:35.262785 3316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:35.263935 3316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:35.263974 3316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:35.263999 3316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:07:35.265388 3316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:40049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:35.265616 3316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:35.265914 3316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:07:35.271662 3316 server_base.cc:1034] running on GCE node
W20260620 23:07:35.271632 3322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:35.271618 3321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:35.271822 3324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:35.272023 3316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:35.272302 3316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:35.273471 3316 hybrid_clock.cc:648] HybridClock initialized: now 1781996855273458 us; error 30 us; skew 500 ppm
I20260620 23:07:35.274708 3316 webserver.cc:460] Webserver started at http://127.17.188.65:39141/ using document root <none> and password file <none>
I20260620 23:07:35.274933 3316 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:35.275000 3316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:35.275154 3316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:35.276186 3316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "46cfe414c63845bca74812c845eecc51"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "1ff61b35835284ee8807515114a5cfe8"
server_key_iv: "f6bda6f1c693b47a55e6b6add626d37a"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.276557 3316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "46cfe414c63845bca74812c845eecc51"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "1ff61b35835284ee8807515114a5cfe8"
server_key_iv: "f6bda6f1c693b47a55e6b6add626d37a"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.277891 3316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:07:35.278620 3330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.278798 3316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:07:35.278892 3316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "46cfe414c63845bca74812c845eecc51"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "1ff61b35835284ee8807515114a5cfe8"
server_key_iv: "f6bda6f1c693b47a55e6b6add626d37a"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.278970 3316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:35.291208 3316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:35.291471 3316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:35.291606 3316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:35.291811 3316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:35.292096 3316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:35.292150 3316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.292196 3316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:35.292220 3316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.297716 3316 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:32977
I20260620 23:07:35.297762 3443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:32977 every 8 connection(s)
I20260620 23:07:35.298060 3316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:07:35.302503 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 3316
I20260620 23:07:35.302613 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:07:35.302846 18161 external_mini_cluster.cc:1428] Setting key 35dc311fa978aec4a22d7b7b3e8fe5c2
I20260620 23:07:35.303212 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.298638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:41585 (local address 127.17.188.126:42809)
0620 23:07:35.298676 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.298678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.298953 (+ 275us) server_negotiation.cc:407] Connection header received
0620 23:07:35.299462 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.299463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.299475 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.299523 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:35.300187 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.300680 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.301699 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.301915 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.301988 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.302897 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:35.302898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:35.302899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:35.302904 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:35.302939 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:35.302944 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:35.302993 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.303104 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.303136 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:07:35.303433 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.298628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42809 (local address 127.17.188.65:41585)
0620 23:07:35.298836 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:35.298878 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:35.299356 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:35.299586 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:35.299590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:35.299599 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:35.300091 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.300095 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.300787 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.300788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.301613 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.301615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.302006 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.302007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.302129 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.302728 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:07:35.302728 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:35.302755 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:07:35.302785 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:35.303024 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:35.303029 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:35.303031 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:07:35.303106 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 23:07:35.303204 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":89,"threads_started":1}
I20260620 23:07:35.303864 3444 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42809
I20260620 23:07:35.303908 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42809
--builtin_ntp_servers=127.17.188.84:40049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:07:35.303966 3444 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:35.304152 3444 heartbeater.cc:511] Master 127.17.188.126:42809 requested a full tablet report, sending...
I20260620 23:07:35.304639 3256 ts_manager.cc:194] Registered new tserver with Master: 46cfe414c63845bca74812c845eecc51 (127.17.188.65:32977)
I20260620 23:07:35.305190 3256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.65:41585
W20260620 23:07:35.379999 3447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:35.380213 3447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:35.380246 3447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:35.381366 3447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:35.381422 3447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:35.381458 3447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:07:35.382699 3447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:40049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:35.382975 3447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:35.383246 3447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:35.387874 3452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:35.387930 3453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:35.388037 3455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:35.387889 3447 server_base.cc:1034] running on GCE node
I20260620 23:07:35.388350 3447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:35.388602 3447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:35.389767 3447 hybrid_clock.cc:648] HybridClock initialized: now 1781996855389765 us; error 51 us; skew 500 ppm
I20260620 23:07:35.391039 3447 webserver.cc:460] Webserver started at http://127.17.188.66:34401/ using document root <none> and password file <none>
I20260620 23:07:35.391273 3447 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:35.391340 3447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:35.391453 3447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:35.392416 3447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "6db672c2ebb94601aee979864e0c05e3"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "f196a9e60fd0bffa9505429196fa59d8"
server_key_iv: "93c452194a7580c3cc596ac8dcd523ae"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.392684 3447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "6db672c2ebb94601aee979864e0c05e3"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "f196a9e60fd0bffa9505429196fa59d8"
server_key_iv: "93c452194a7580c3cc596ac8dcd523ae"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.393865 3447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:07:35.394482 3461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.394660 3447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:35.394729 3447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "6db672c2ebb94601aee979864e0c05e3"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "f196a9e60fd0bffa9505429196fa59d8"
server_key_iv: "93c452194a7580c3cc596ac8dcd523ae"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.394778 3447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:35.402812 3447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:35.403050 3447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:35.403139 3447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:35.403298 3447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:35.403515 3447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:35.403542 3447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.403561 3447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:35.403573 3447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.408986 3447 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:32791
I20260620 23:07:35.409054 3574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:32791 every 8 connection(s)
I20260620 23:07:35.409359 3447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:07:35.413993 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.410045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:56005 (local address 127.17.188.126:42809)
0620 23:07:35.410110 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.410111 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.410317 (+ 206us) server_negotiation.cc:407] Connection header received
0620 23:07:35.410732 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.410732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.410745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.410799 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:35.411382 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.411917 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.412827 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.413004 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.413055 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.413790 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:35.413791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:35.413791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:35.413795 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:35.413807 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:35.413810 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:35.413856 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.413916 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.413939 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:07:35.414197 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.410033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42809 (local address 127.17.188.66:56005)
0620 23:07:35.410197 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:35.410233 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:35.410656 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:35.410845 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:35.410848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:35.410855 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:35.411312 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.411315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.411987 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.411987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.412728 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.412730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.413073 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.413074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.413172 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.413673 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:07:35.413674 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:35.413699 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:07:35.413718 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:35.413858 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:35.413860 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:35.413862 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:07:35.413906 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:07:35.413973 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260620 23:07:35.414616 3575 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42809
I20260620 23:07:35.414710 3575 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:35.414889 3575 heartbeater.cc:511] Master 127.17.188.126:42809 requested a full tablet report, sending...
I20260620 23:07:35.415220 3256 ts_manager.cc:194] Registered new tserver with Master: 6db672c2ebb94601aee979864e0c05e3 (127.17.188.66:32791)
I20260620 23:07:35.415599 3256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.66:56005
I20260620 23:07:35.418712 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 3447
I20260620 23:07:35.418783 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:07:35.418926 18161 external_mini_cluster.cc:1428] Setting key dbbc83cc25fa95d0bf2f68bbbcd073f2
I20260620 23:07:35.419703 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42809
--builtin_ntp_servers=127.17.188.84:40049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:07:35.498423 3578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:35.498600 3578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:35.498621 3578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:35.499635 3578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:35.499665 3578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:35.499687 3578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:07:35.500934 3578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:40049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:42809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:35.501163 3578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:35.501407 3578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:35.506029 3583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:35.506187 3586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:35.506237 3578 server_base.cc:1034] running on GCE node
W20260620 23:07:35.506275 3584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:35.506579 3578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:35.506807 3578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:35.507956 3578 hybrid_clock.cc:648] HybridClock initialized: now 1781996855507933 us; error 39 us; skew 500 ppm
I20260620 23:07:35.509166 3578 webserver.cc:460] Webserver started at http://127.17.188.67:41847/ using document root <none> and password file <none>
I20260620 23:07:35.509462 3578 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:35.509531 3578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:35.509665 3578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:35.510687 3578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "863de169ad7d4412a7c9547b64bcf57c"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "c2bc68256bc3373d78ce52e42ad1e82b"
server_key_iv: "03cfaaa3d2bca1fcb98bc1d8e18d9271"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.511065 3578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "863de169ad7d4412a7c9547b64bcf57c"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "c2bc68256bc3373d78ce52e42ad1e82b"
server_key_iv: "03cfaaa3d2bca1fcb98bc1d8e18d9271"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.512356 3578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:07:35.513012 3592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.513164 3578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:35.513254 3578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "863de169ad7d4412a7c9547b64bcf57c"
format_stamp: "Formatted at 2026-06-20 23:07:35 on dist-test-slave-zpss"
server_key: "c2bc68256bc3373d78ce52e42ad1e82b"
server_key_iv: "03cfaaa3d2bca1fcb98bc1d8e18d9271"
server_key_version: "encryptionkey@0"
I20260620 23:07:35.513360 3578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:35.525178 3578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:35.525499 3578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:35.525635 3578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:35.525821 3578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:35.526098 3578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:35.526154 3578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.526189 3578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:35.526213 3578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:35.531584 3578 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:35609
I20260620 23:07:35.531649 3705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:35609 every 8 connection(s)
I20260620 23:07:35.531956 3578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:07:35.534133 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 3578
I20260620 23:07:35.534240 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:07:35.534409 18161 external_mini_cluster.cc:1428] Setting key e896420f41e91d1752e478ce00fbc201
I20260620 23:07:35.537047 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.532639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:33889 (local address 127.17.188.126:42809)
0620 23:07:35.532699 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.532700 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.532883 (+ 183us) server_negotiation.cc:407] Connection header received
0620 23:07:35.533270 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.533271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.533280 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.533394 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:35.534056 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.534710 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.535930 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.536127 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.536185 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.536841 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:35.536842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:35.536842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:35.536846 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:35.536859 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:35.536862 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:35.536908 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.536972 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.536991 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:07:35.537240 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.532611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42809 (local address 127.17.188.67:33889)
0620 23:07:35.532771 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:35.532805 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:35.533191 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:35.533470 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:35.533474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:35.533482 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:35.533976 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.533981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.534849 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.534850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.535856 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.535858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.536199 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.536200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.536307 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.536725 (+ 418us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:07:35.536725 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:35.536750 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:07:35.536770 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:35.536911 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:35.536913 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:35.536915 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:07:35.536960 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 23:07:35.537026 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260620 23:07:35.537673 3706 heartbeater.cc:347] Connected to a master server at 127.17.188.126:42809
I20260620 23:07:35.537755 3706 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:35.537875 3706 heartbeater.cc:511] Master 127.17.188.126:42809 requested a full tablet report, sending...
I20260620 23:07:35.538182 3256 ts_manager.cc:194] Registered new tserver with Master: 863de169ad7d4412a7c9547b64bcf57c (127.17.188.67:35609)
I20260620 23:07:35.538522 3256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.67:33889
I20260620 23:07:35.546025 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:07:35.549820 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.547287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.17.188.126:42809)
0620 23:07:35.547342 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.547344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.547478 (+ 134us) server_negotiation.cc:407] Connection header received
0620 23:07:35.547527 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.547528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.547535 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.547581 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:35.547721 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.548116 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.548839 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.548999 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.549044 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.549621 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:35.549622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:35.549622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:35.549626 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:35.549637 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:35.549641 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:35.549684 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.549731 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.549747 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:07:35.551327 3256 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:07:35.552697 3256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:07:35.558314 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.554609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:32977 (local address 127.0.0.1:60100)
0620 23:07:35.555076 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:35.555084 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:35.555131 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:35.555378 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:35.555378 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:35.555380 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:35.555414 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.555415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.556252 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.556253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.557259 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.557261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.558025 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.558025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.558093 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.558123 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 23:07:35.558195 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 23:07:35.558245 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260620 23:07:35.558437 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.554612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:35609 (local address 127.0.0.1:49922)
0620 23:07:35.555188 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:35.555196 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:35.555235 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:35.555469 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:35.555470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:35.555472 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:35.555520 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.555522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.556572 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.556573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.557233 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.557235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.558171 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.558172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.558237 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.558241 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:07:35.558319 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 23:07:35.558359 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260620 23:07:35.558545 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.554601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:32791 (local address 127.0.0.1:47232)
0620 23:07:35.554867 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:35.554903 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:35.554973 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:35.555309 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:35.555311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:35.555313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:35.555370 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.555372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.556543 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.556544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.557460 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.557462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.558348 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.558349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.558399 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.558402 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:35.558444 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 23:07:35.558479 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":19,"thread_start_us":67,"threads_started":1}
I20260620 23:07:35.558655 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.554804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49922 (local address 127.17.188.67:35609)
0620 23:07:35.554947 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.554952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.555266 (+ 314us) server_negotiation.cc:407] Connection header received
0620 23:07:35.555299 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.555301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.555313 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.555375 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:35.555583 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.556507 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.557333 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.557979 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.558043 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.558135 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.558540 (+ 405us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.558585 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260620 23:07:35.558336 3719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.554818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60100 (local address 127.17.188.65:32977)
0620 23:07:35.555000 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.555003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.555142 (+ 139us) server_negotiation.cc:407] Connection header received
0620 23:07:35.555191 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.555192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.555203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.555288 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:35.555520 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.556172 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.557447 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.557936 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.558013 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.558093 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.558219 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.558256 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260620 23:07:35.558930 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.554683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.17.188.66:32791)
0620 23:07:35.554982 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.554986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.554996 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:07:35.555057 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.555060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.555075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.555146 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:35.555449 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.556465 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.557528 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.558183 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.558240 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.558316 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.558621 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.558662 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":84,"threads_started":1}
I20260620 23:07:35.559355 3640 tablet_service.cc:1467] Processing CreateTablet for tablet 58df0431bd234ebcaf9af5ddac8e17d7 (DEFAULT_TABLE table=test-table [id=5796e52e866f4fd8865614621ce68500]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:35.559367 3378 tablet_service.cc:1467] Processing CreateTablet for tablet 58df0431bd234ebcaf9af5ddac8e17d7 (DEFAULT_TABLE table=test-table [id=5796e52e866f4fd8865614621ce68500]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:35.559507 3509 tablet_service.cc:1467] Processing CreateTablet for tablet 58df0431bd234ebcaf9af5ddac8e17d7 (DEFAULT_TABLE table=test-table [id=5796e52e866f4fd8865614621ce68500]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:35.559667 3640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58df0431bd234ebcaf9af5ddac8e17d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:35.559691 3378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58df0431bd234ebcaf9af5ddac8e17d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:35.559764 3509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58df0431bd234ebcaf9af5ddac8e17d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:35.562085 3720 tablet_bootstrap.cc:492] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51: Bootstrap starting.
I20260620 23:07:35.562597 3720 tablet_bootstrap.cc:654] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:35.562855 3722 tablet_bootstrap.cc:492] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: Bootstrap starting.
I20260620 23:07:35.562937 3720 log.cc:824] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:35.563437 3722 tablet_bootstrap.cc:654] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:35.563755 3722 log.cc:824] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:35.563779 3720 tablet_bootstrap.cc:492] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51: No bootstrap required, opened a new log
I20260620 23:07:35.563844 3720 ts_tablet_manager.cc:1397] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:35.564093 3721 tablet_bootstrap.cc:492] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: Bootstrap starting.
I20260620 23:07:35.564399 3722 tablet_bootstrap.cc:492] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: No bootstrap required, opened a new log
I20260620 23:07:35.564474 3722 ts_tablet_manager.cc:1397] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:35.564744 3721 tablet_bootstrap.cc:654] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:35.565035 3721 log.cc:824] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:35.565783 3721 tablet_bootstrap.cc:492] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: No bootstrap required, opened a new log
I20260620 23:07:35.565845 3721 ts_tablet_manager.cc:1397] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:35.566588 3722 raft_consensus.cc:348] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } }
I20260620 23:07:35.566543 3720 raft_consensus.cc:348] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } }
I20260620 23:07:35.566727 3722 raft_consensus.cc:374] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:35.566726 3720 raft_consensus.cc:374] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:35.566773 3722 raft_consensus.cc:729] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6db672c2ebb94601aee979864e0c05e3, State: Initialized, Role: FOLLOWER
I20260620 23:07:35.566780 3720 raft_consensus.cc:729] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46cfe414c63845bca74812c845eecc51, State: Initialized, Role: FOLLOWER
I20260620 23:07:35.566881 3722 consensus_queue.cc:260] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } }
I20260620 23:07:35.566926 3720 consensus_queue.cc:260] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } }
I20260620 23:07:35.567157 3722 ts_tablet_manager.cc:1428] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:07:35.567173 3720 ts_tablet_manager.cc:1428] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260620 23:07:35.567207 3444 heartbeater.cc:503] Master 127.17.188.126:42809 was elected leader, sending a full tablet report...
I20260620 23:07:35.567289 3575 heartbeater.cc:503] Master 127.17.188.126:42809 was elected leader, sending a full tablet report...
I20260620 23:07:35.568523 3721 raft_consensus.cc:348] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } }
I20260620 23:07:35.568624 3721 raft_consensus.cc:374] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:35.568641 3721 raft_consensus.cc:729] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863de169ad7d4412a7c9547b64bcf57c, State: Initialized, Role: FOLLOWER
I20260620 23:07:35.568749 3721 consensus_queue.cc:260] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } }
I20260620 23:07:35.568949 3721 ts_tablet_manager.cc:1428] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:07:35.569007 3706 heartbeater.cc:503] Master 127.17.188.126:42809 was elected leader, sending a full tablet report...
W20260620 23:07:35.659989 3576 tablet.cc:2369] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:07:35.782455 3707 tablet.cc:2369] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:07:35.798647 3445 tablet.cc:2369] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:07:35.963675 3727 raft_consensus.cc:482] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:35.963793 3727 raft_consensus.cc:504] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } }
I20260620 23:07:35.964147 3727 leader_election.cc:288] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6db672c2ebb94601aee979864e0c05e3 (127.17.188.66:32791), 863de169ad7d4412a7c9547b64bcf57c (127.17.188.67:35609)
I20260620 23:07:35.968115 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.964373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:39437 (local address 127.17.188.66:32791)
0620 23:07:35.964431 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.964434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.964612 (+ 178us) server_negotiation.cc:407] Connection header received
0620 23:07:35.964660 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.964661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.964671 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.964718 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:35.964914 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.965656 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.967213 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.967758 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.967835 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.967843 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.968012 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.968047 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:07:35.968115 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.964380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:36007 (local address 127.17.188.67:35609)
0620 23:07:35.964431 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.964434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.964531 (+ 97us) server_negotiation.cc:407] Connection header received
0620 23:07:35.964624 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.964625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.964636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.964695 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:35.964915 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.965652 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.967212 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.967767 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.967835 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.967843 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.968010 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.968047 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:07:35.968163 3730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.964310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:32791 (local address 127.17.188.65:39437)
0620 23:07:35.964542 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:35.964551 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:35.964599 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:35.964732 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:35.964732 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:35.964733 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:35.964844 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.964847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.965733 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.965734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.967116 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.967120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.967855 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.967856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.967925 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.967932 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 23:07:35.967989 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:07:35.968046 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":65,"thread_start_us":59,"threads_started":1}
I20260620 23:07:35.968364 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.964307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:35609 (local address 127.17.188.65:36007)
0620 23:07:35.964438 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:35.964450 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:35.964540 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:35.964724 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:35.964725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:35.964727 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:35.964844 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.964846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.965734 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.965735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.967116 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:35.967120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.967843 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.967845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:35.967925 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.967931 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:07:35.967989 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 23:07:35.968046 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":104,"thread_start_us":64,"threads_started":1}
I20260620 23:07:35.968464 3529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58df0431bd234ebcaf9af5ddac8e17d7" candidate_uuid: "46cfe414c63845bca74812c845eecc51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6db672c2ebb94601aee979864e0c05e3" is_pre_election: true
I20260620 23:07:35.968607 3529 raft_consensus.cc:2454] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46cfe414c63845bca74812c845eecc51 in term 0.
I20260620 23:07:35.968670 3660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58df0431bd234ebcaf9af5ddac8e17d7" candidate_uuid: "46cfe414c63845bca74812c845eecc51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "863de169ad7d4412a7c9547b64bcf57c" is_pre_election: true
I20260620 23:07:35.968817 3660 raft_consensus.cc:2454] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46cfe414c63845bca74812c845eecc51 in term 0.
I20260620 23:07:35.968847 3334 leader_election.cc:302] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46cfe414c63845bca74812c845eecc51, 6db672c2ebb94601aee979864e0c05e3; no voters:
I20260620 23:07:35.968959 3727 raft_consensus.cc:2781] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:07:35.969017 3727 raft_consensus.cc:482] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:07:35.969033 3727 raft_consensus.cc:3037] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:35.969816 3727 raft_consensus.cc:504] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } }
I20260620 23:07:35.969956 3727 leader_election.cc:288] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [CANDIDATE]: Term 1 election: Requested vote from peers 6db672c2ebb94601aee979864e0c05e3 (127.17.188.66:32791), 863de169ad7d4412a7c9547b64bcf57c (127.17.188.67:35609)
I20260620 23:07:35.970120 3529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58df0431bd234ebcaf9af5ddac8e17d7" candidate_uuid: "46cfe414c63845bca74812c845eecc51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6db672c2ebb94601aee979864e0c05e3"
I20260620 23:07:35.970132 3660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58df0431bd234ebcaf9af5ddac8e17d7" candidate_uuid: "46cfe414c63845bca74812c845eecc51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "863de169ad7d4412a7c9547b64bcf57c"
I20260620 23:07:35.970201 3660 raft_consensus.cc:3037] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:35.970201 3529 raft_consensus.cc:3037] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:35.971051 3660 raft_consensus.cc:2454] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46cfe414c63845bca74812c845eecc51 in term 1.
I20260620 23:07:35.971051 3529 raft_consensus.cc:2454] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46cfe414c63845bca74812c845eecc51 in term 1.
I20260620 23:07:35.971225 3334 leader_election.cc:302] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46cfe414c63845bca74812c845eecc51, 6db672c2ebb94601aee979864e0c05e3; no voters:
I20260620 23:07:35.971335 3727 raft_consensus.cc:2781] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:35.971503 3727 raft_consensus.cc:686] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 1 LEADER]: Becoming Leader. State: Replica: 46cfe414c63845bca74812c845eecc51, State: Running, Role: LEADER
I20260620 23:07:35.971596 3727 consensus_queue.cc:237] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } }
I20260620 23:07:35.972414 3255 catalog_manager.cc:5495] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46cfe414c63845bca74812c845eecc51 (127.17.188.65). New cstate: current_term: 1 leader_uuid: "46cfe414c63845bca74812c845eecc51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46cfe414c63845bca74812c845eecc51" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 32977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 } health_report { overall_health: UNKNOWN } } }
I20260620 23:07:35.998972 3719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:35.996744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.17.188.65:32977)
0620 23:07:35.996803 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:35.996805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:35.996812 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:07:35.996871 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:35.996871 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:35.996887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:35.996939 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:07:35.997028 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.997581 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.998361 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:35.998531 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:35.998583 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:35.998698 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:07:35.998782 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:07:35.998849 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:35.998878 (+ 29us) server_negotiation.cc:299] Negotiation successful
0620 23:07:35.998894 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:07:36.000540 3529 raft_consensus.cc:1261] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [term 1 FOLLOWER]: Refusing update from remote peer 46cfe414c63845bca74812c845eecc51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:36.000556 3660 raft_consensus.cc:1261] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [term 1 FOLLOWER]: Refusing update from remote peer 46cfe414c63845bca74812c845eecc51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:36.000900 3727 consensus_queue.cc:1035] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6db672c2ebb94601aee979864e0c05e3" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:36.001041 3731 consensus_queue.cc:1035] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "863de169ad7d4412a7c9547b64bcf57c" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 35609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:36.003597 3736 mvcc.cc:205] Tried to move back new op lower bound from 7299059122175893504 to 7299059122060255232. Current Snapshot: MvccSnapshot[applied={T|T < 7299059122175893504}]
I20260620 23:07:36.005266 3256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:36.005391 3256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:36.006136 3256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b9e5334e2174203849f39a7259f4f4e: Sending DeleteTablet for 3 replicas of tablet 58df0431bd234ebcaf9af5ddac8e17d7
I20260620 23:07:36.006474 3640 tablet_service.cc:1514] Processing DeleteTablet for tablet 58df0431bd234ebcaf9af5ddac8e17d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:36 UTC) from {username='slave'} at 127.0.0.1:49922
I20260620 23:07:36.006475 3378 tablet_service.cc:1514] Processing DeleteTablet for tablet 58df0431bd234ebcaf9af5ddac8e17d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:36 UTC) from {username='slave'} at 127.0.0.1:60100
I20260620 23:07:36.006498 3509 tablet_service.cc:1514] Processing DeleteTablet for tablet 58df0431bd234ebcaf9af5ddac8e17d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:36 UTC) from {username='slave'} at 127.0.0.1:47232
I20260620 23:07:36.006668 3744 tablet_replica.cc:330] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: stopping tablet replica
I20260620 23:07:36.006755 3744 raft_consensus.cc:2229] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:36.006857 3744 raft_consensus.cc:2258] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:36.006959 3743 tablet_replica.cc:330] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51: stopping tablet replica
I20260620 23:07:36.006989 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 3316
I20260620 23:07:36.006880 3745 tablet_replica.cc:330] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: stopping tablet replica
I20260620 23:07:36.007180 3745 raft_consensus.cc:2229] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:36.007318 3743 raft_consensus.cc:2229] T 58df0431bd234ebcaf9af5ddac8e17d7 P 46cfe414c63845bca74812c845eecc51 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:07:36.007347 3744 ts_tablet_manager.cc:1905] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:36.007601 3745 raft_consensus.cc:2258] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:36.008440 3745 ts_tablet_manager.cc:1905] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:36.009243 3744 ts_tablet_manager.cc:1918] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:36.009346 3744 log.cc:1196] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/58df0431bd234ebcaf9af5ddac8e17d7
I20260620 23:07:36.009626 3744 ts_tablet_manager.cc:1939] T 58df0431bd234ebcaf9af5ddac8e17d7 P 863de169ad7d4412a7c9547b64bcf57c: Deleting consensus metadata
I20260620 23:07:36.009999 3242 catalog_manager.cc:4841] TS 863de169ad7d4412a7c9547b64bcf57c (127.17.188.67:35609): tablet 58df0431bd234ebcaf9af5ddac8e17d7 (table test-table [id=5796e52e866f4fd8865614621ce68500]) successfully deleted
I20260620 23:07:36.010355 3745 ts_tablet_manager.cc:1918] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:36.010442 3745 log.cc:1196] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/58df0431bd234ebcaf9af5ddac8e17d7
I20260620 23:07:36.010659 3745 ts_tablet_manager.cc:1939] T 58df0431bd234ebcaf9af5ddac8e17d7 P 6db672c2ebb94601aee979864e0c05e3: Deleting consensus metadata
I20260620 23:07:36.011056 3243 catalog_manager.cc:4841] TS 6db672c2ebb94601aee979864e0c05e3 (127.17.188.66:32791): tablet 58df0431bd234ebcaf9af5ddac8e17d7 (table test-table [id=5796e52e866f4fd8865614621ce68500]) successfully deleted
W20260620 23:07:36.012296 3241 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:32977 (error 108)
I20260620 23:07:36.012800 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:36.012638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:32977 (local address 127.0.0.1:60110)
0620 23:07:36.012697 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:36.012759 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:32977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 23:07:36.012835 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 3447
W20260620 23:07:36.012943 3241 catalog_manager.cc:4584] TS 46cfe414c63845bca74812c845eecc51 (127.17.188.65:32977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58df0431bd234ebcaf9af5ddac8e17d7: Network error: Client connection negotiation failed: client connection to 127.17.188.65:32977: connect: Connection refused (error 111)
I20260620 23:07:36.017580 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 3578
I20260620 23:07:36.022521 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 3225
2026-06-20T23:07:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1037 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:36 dist-test-slave-zpss krb5kdc[3749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:36 dist-test-slave-zpss krb5kdc[3749](info): set up 2 sockets
Jun 20 23:07:36 dist-test-slave-zpss krb5kdc[3749](info): commencing operation
krb5kdc: starting...
W20260620 23:07:37.025045 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:37 dist-test-slave-zpss krb5kdc[3749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:07:37.133672 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35299
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:35299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:37.210460 3762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:37.210639 3762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:37.210668 3762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:37.211704 3762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:37.211735 3762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:37.211748 3762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:37.211760 3762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:37.211772 3762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:37.213078 3762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:35299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:37.213240 3762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:37.213415 3762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:07:37.217862 3762 server_base.cc:1034] running on GCE node
W20260620 23:07:37.217820 3770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:37.217854 3767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:37.217835 3768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:37.218276 3762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:37.218523 3762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:37.219658 3762 hybrid_clock.cc:648] HybridClock initialized: now 1781996857219647 us; error 27 us; skew 500 ppm
Jun 20 23:07:37 dist-test-slave-zpss krb5kdc[3749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996857, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:37.222007 3762 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:07:37.222415 3762 webserver.cc:460] Webserver started at http://127.17.188.126:33569/ using document root <none> and password file <none>
I20260620 23:07:37.222590 3762 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:37.222628 3762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:37.222707 3762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:37.223641 3762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "1861d43c6e334ad99bbd1dde1e290a61"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "f479024870143e5e1c04a68524f3f335"
server_key_iv: "ec7f9e07a0bb020a06ab71a565597469"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.223932 3762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "1861d43c6e334ad99bbd1dde1e290a61"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "f479024870143e5e1c04a68524f3f335"
server_key_iv: "ec7f9e07a0bb020a06ab71a565597469"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.225019 3762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:07:37.225692 3777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.225867 3762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.225947 3762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "1861d43c6e334ad99bbd1dde1e290a61"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "f479024870143e5e1c04a68524f3f335"
server_key_iv: "ec7f9e07a0bb020a06ab71a565597469"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.225996 3762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:37.241837 3762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:37.243497 3762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:37.243598 3762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:37.247198 3762 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:35299
I20260620 23:07:37.247257 3829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:35299 every 8 connection(s)
I20260620 23:07:37.247588 3762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:37.248389 3830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:37.250442 3830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61: Bootstrap starting.
I20260620 23:07:37.250967 3830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:37.251251 3830 log.cc:824] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:37.251921 3830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61: No bootstrap required, opened a new log
I20260620 23:07:37.253500 3830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1861d43c6e334ad99bbd1dde1e290a61" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35299 } }
I20260620 23:07:37.253643 3830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:37.253686 3830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1861d43c6e334ad99bbd1dde1e290a61, State: Initialized, Role: FOLLOWER
I20260620 23:07:37.253865 3830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1861d43c6e334ad99bbd1dde1e290a61" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35299 } }
I20260620 23:07:37.253974 3830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:37.254017 3830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:37.254067 3830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:37.254532 3830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1861d43c6e334ad99bbd1dde1e290a61" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35299 } }
I20260620 23:07:37.254642 3830 leader_election.cc:302] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1861d43c6e334ad99bbd1dde1e290a61; no voters:
I20260620 23:07:37.254817 3830 leader_election.cc:288] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:37.254850 3833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:37.255019 3833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [term 1 LEADER]: Becoming Leader. State: Replica: 1861d43c6e334ad99bbd1dde1e290a61, State: Running, Role: LEADER
I20260620 23:07:37.255067 3830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:37.255170 3833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1861d43c6e334ad99bbd1dde1e290a61" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35299 } }
I20260620 23:07:37.255545 3835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1861d43c6e334ad99bbd1dde1e290a61. Latest consensus state: current_term: 1 leader_uuid: "1861d43c6e334ad99bbd1dde1e290a61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1861d43c6e334ad99bbd1dde1e290a61" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35299 } } }
I20260620 23:07:37.255672 3834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1861d43c6e334ad99bbd1dde1e290a61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1861d43c6e334ad99bbd1dde1e290a61" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35299 } } }
I20260620 23:07:37.255752 3834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:37.255718 3835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:37.256174 3842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:37.256650 3842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:37.257504 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 3762
I20260620 23:07:37.257619 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:37.257813 18161 external_mini_cluster.cc:1428] Setting key de5328625a3e1474362e8caf0ed9d91f
I20260620 23:07:37.258781 3842 catalog_manager.cc:1347] Generated new cluster ID: 8fb381c0b0e040bbaa10ffe453f7f06f
I20260620 23:07:37.258848 3842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:07:37 dist-test-slave-zpss krb5kdc[3749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:37.269135 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.258360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.17.188.126:35299)
0620 23:07:37.258842 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.258849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.258877 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:07:37.258990 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.258996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.259046 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.259178 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:37.259830 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.261275 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.262286 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.262964 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.263076 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.265688 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:37.265692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:37.265692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:37.265703 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:37.267619 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.267986 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.267988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.267989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.268089 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.268274 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.268275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.268275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.268551 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:37.268700 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.268775 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.268898 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":70,"threads_started":1}
I20260620 23:07:37.283200 3842 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:37.283686 3842 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:37.290467 3842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61: Generated new TSK 0
I20260620 23:07:37.290625 3842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:07:37.324463 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35299
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:37.419526 3856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:37.419690 3856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:37.419710 3856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:37.420799 3856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:37.420830 3856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:37.420856 3856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:07:37.422233 3856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:37.422400 3856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:37.422546 3856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:37.427184 3862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:37.427204 3861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:37.427206 3864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:37.427301 3856 server_base.cc:1034] running on GCE node
I20260620 23:07:37.427604 3856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:37.427824 3856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:37.428985 3856 hybrid_clock.cc:648] HybridClock initialized: now 1781996857428963 us; error 35 us; skew 500 ppm
Jun 20 23:07:37 dist-test-slave-zpss krb5kdc[3749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996857, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:37.431322 3856 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:07:37.431751 3856 webserver.cc:460] Webserver started at http://127.17.188.65:42101/ using document root <none> and password file <none>
I20260620 23:07:37.431967 3856 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:37.432026 3856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:37.432160 3856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:37.433153 3856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "66ac337f97014c3aa6ccc655f21399dc"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "469e432bbc8a76904298c932d7918c99"
server_key_iv: "8251e4916892de7cba9bcfd79e7f63c3"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.433544 3856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "66ac337f97014c3aa6ccc655f21399dc"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "469e432bbc8a76904298c932d7918c99"
server_key_iv: "8251e4916892de7cba9bcfd79e7f63c3"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.434845 3856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:07:37.435483 3871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.435657 3856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:37.435750 3856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "66ac337f97014c3aa6ccc655f21399dc"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "469e432bbc8a76904298c932d7918c99"
server_key_iv: "8251e4916892de7cba9bcfd79e7f63c3"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.435828 3856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:37.454485 3856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:37.456388 3856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:37.456548 3856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:37.456765 3856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:37.457135 3856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:37.457175 3856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.457222 3856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:37.457243 3856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.464437 3856 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:45951
I20260620 23:07:37.464471 3984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:45951 every 8 connection(s)
I20260620 23:07:37.464761 3856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:07:37.469696 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 3856
I20260620 23:07:37.469789 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:07:37.469992 18161 external_mini_cluster.cc:1428] Setting key 6cb4690196a05cba68b2e318fdbba6b3
Jun 20 23:07:37 dist-test-slave-zpss krb5kdc[3749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996857, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:37.473537 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.465479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:49781 (local address 127.17.188.126:35299)
0620 23:07:37.465538 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.465540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.465758 (+ 218us) server_negotiation.cc:407] Connection header received
0620 23:07:37.466174 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.466175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.466188 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.466236 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:37.467015 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.467479 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.468499 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.468685 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.468733 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.470910 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:37.470911 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:37.470912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:37.470919 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:37.472245 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.472621 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.472622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.472622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.472677 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.472921 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.472922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.472923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.473025 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:37.473097 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.473372 (+ 275us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.473451 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:07:37.473716 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.465479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35299 (local address 127.17.188.65:49781)
0620 23:07:37.465638 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:37.465670 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:37.466096 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:37.466283 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:37.466286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:37.466778 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:37.466944 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.466948 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.467655 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.467656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.468424 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.468426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.468767 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.468768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.468859 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.469260 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:37.469267 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:37.470819 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:37.472332 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:37.472339 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:37.472341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:37.472547 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:37.472742 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:37.472743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:37.472743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:37.472849 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:37.473090 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:37.473096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:37.473225 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 23:07:37.473357 (+ 132us) client_negotiation.cc:240] Negotiation successful
0620 23:07:37.473501 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260620 23:07:37.474269 3985 heartbeater.cc:347] Connected to a master server at 127.17.188.126:35299
I20260620 23:07:37.474345 3985 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:37.474514 3985 heartbeater.cc:511] Master 127.17.188.126:35299 requested a full tablet report, sending...
I20260620 23:07:37.475083 3794 ts_manager.cc:194] Registered new tserver with Master: 66ac337f97014c3aa6ccc655f21399dc (127.17.188.65:45951)
I20260620 23:07:37.476001 3794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:49781
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:07:37.492142 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35299
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:37.569242 3990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:37.569475 3990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:37.569500 3990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:37.570591 3990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:37.570626 3990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:37.570652 3990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:07:37.571978 3990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:37.572170 3990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:37.572331 3990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:07:37.577040 3990 server_base.cc:1034] running on GCE node
W20260620 23:07:37.576995 3996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:37.576984 3995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:37.576995 3998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:37.577450 3990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:37.577735 3990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:37.578897 3990 hybrid_clock.cc:648] HybridClock initialized: now 1781996857578883 us; error 26 us; skew 500 ppm
Jun 20 23:07:37 dist-test-slave-zpss krb5kdc[3749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996857, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:37.581163 3990 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:07:37.581656 3990 webserver.cc:460] Webserver started at http://127.17.188.66:36033/ using document root <none> and password file <none>
I20260620 23:07:37.581820 3990 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:37.581858 3990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:37.581936 3990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:37.582834 3990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "c2261d86593249d1907ddb16350d507c"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "53bbab20ec41ee85961cc4ed0995f0c9"
server_key_iv: "33aab662cffd8500427749b28402c335"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.583134 3990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "c2261d86593249d1907ddb16350d507c"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "53bbab20ec41ee85961cc4ed0995f0c9"
server_key_iv: "33aab662cffd8500427749b28402c335"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.584316 3990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:07:37.584911 4005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.585050 3990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.585122 3990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "c2261d86593249d1907ddb16350d507c"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "53bbab20ec41ee85961cc4ed0995f0c9"
server_key_iv: "33aab662cffd8500427749b28402c335"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.585170 3990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:37.594681 3990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:37.596736 3990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:37.596837 3990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:37.597007 3990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:37.597219 3990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:37.597245 3990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.597263 3990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:37.597277 3990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.602497 3990 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:40349
I20260620 23:07:37.602591 4118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:40349 every 8 connection(s)
I20260620 23:07:37.602886 3990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:07:37.606045 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 3990
I20260620 23:07:37.606155 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:07:37.606355 18161 external_mini_cluster.cc:1428] Setting key 7991810ac66bc4afbc36eec723bfdae3
Jun 20 23:07:37 dist-test-slave-zpss krb5kdc[3749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996857, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:37.612636 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.603571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:37467 (local address 127.17.188.126:35299)
0620 23:07:37.603614 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.603620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.603792 (+ 172us) server_negotiation.cc:407] Connection header received
0620 23:07:37.604224 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.604225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.604233 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.604282 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:37.605079 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.605625 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.607031 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.607294 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.607347 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.609935 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:37.609936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:37.609937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:37.609945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:37.611343 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.611745 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.611746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.611747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.611806 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.612053 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.612054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.612054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.612155 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:37.612218 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.612461 (+ 243us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.612533 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:07:37.612821 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.603527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35299 (local address 127.17.188.66:37467)
0620 23:07:37.603682 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:37.603720 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:37.604149 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:37.604334 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:37.604336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:37.604842 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:37.605009 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.605013 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.605700 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.605701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.606947 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.606950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.607375 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.607376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.607516 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.607978 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:37.607987 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:37.609844 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:37.611454 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:37.611461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:37.611463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:37.611672 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:37.611882 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:37.611883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:37.611883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:37.611987 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:37.612221 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:37.612227 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:37.612346 (+ 119us) client_negotiation.cc:769] Sending connection context
0620 23:07:37.612442 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 23:07:37.612594 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260620 23:07:37.613441 4119 heartbeater.cc:347] Connected to a master server at 127.17.188.126:35299
I20260620 23:07:37.613515 4119 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:37.613695 4119 heartbeater.cc:511] Master 127.17.188.126:35299 requested a full tablet report, sending...
I20260620 23:07:37.614112 3794 ts_manager.cc:194] Registered new tserver with Master: c2261d86593249d1907ddb16350d507c (127.17.188.66:40349)
I20260620 23:07:37.614471 3794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:37467
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:07:37.629814 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35299
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:46337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:37.705916 4124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:37.706089 4124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:37.706111 4124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:37.707131 4124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:37.707165 4124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:37.707190 4124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:07:37.708493 4124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:46337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:35299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:37.708652 4124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:37.708832 4124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:37.713536 4129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:37.713548 4130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:37.713804 4124 server_base.cc:1034] running on GCE node
W20260620 23:07:37.714006 4132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:37.714200 4124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:37.714452 4124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:37.715608 4124 hybrid_clock.cc:648] HybridClock initialized: now 1781996857715596 us; error 28 us; skew 500 ppm
Jun 20 23:07:37 dist-test-slave-zpss krb5kdc[3749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996857, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:37.717967 4124 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:07:37.718346 4124 webserver.cc:460] Webserver started at http://127.17.188.67:44791/ using document root <none> and password file <none>
I20260620 23:07:37.718550 4124 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:37.718612 4124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:37.718751 4124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:37.719645 4124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "4b1d7c6706c8c406ec56ad6a9e22089e"
server_key_iv: "90d4d4a3d1f21bc6ed70bf1cdec727d2"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.719990 4124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "4b1d7c6706c8c406ec56ad6a9e22089e"
server_key_iv: "90d4d4a3d1f21bc6ed70bf1cdec727d2"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.721217 4124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:07:37.721911 4139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.722066 4124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.722157 4124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e"
format_stamp: "Formatted at 2026-06-20 23:07:37 on dist-test-slave-zpss"
server_key: "4b1d7c6706c8c406ec56ad6a9e22089e"
server_key_iv: "90d4d4a3d1f21bc6ed70bf1cdec727d2"
server_key_version: "encryptionkey@0"
I20260620 23:07:37.722231 4124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:37.729799 4124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:37.731961 4124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:37.732105 4124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:37.732300 4124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:37.732566 4124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:37.732620 4124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.732658 4124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:37.732687 4124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:37.738209 4124 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:40985
I20260620 23:07:37.738252 4252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:40985 every 8 connection(s)
I20260620 23:07:37.738504 4124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:07:37 dist-test-slave-zpss krb5kdc[3749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996857, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:37.743734 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 4124
I20260620 23:07:37.743860 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:07:37.744072 18161 external_mini_cluster.cc:1428] Setting key 6137564d2ce2ee2cc67c8740b40822b4
I20260620 23:07:37.746953 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.739130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:41639 (local address 127.17.188.126:35299)
0620 23:07:37.739200 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.739203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.739433 (+ 230us) server_negotiation.cc:407] Connection header received
0620 23:07:37.739802 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.739822 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.739831 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.739879 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:37.740717 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.741189 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.742144 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.742325 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.742375 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.744271 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:37.744272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:37.744273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:37.744281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:37.745750 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.746086 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.746087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.746087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.746133 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.746347 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.746348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.746348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.746465 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:37.746539 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.746794 (+ 255us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.746868 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 23:07:37.747104 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.739104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:35299 (local address 127.17.188.67:41639)
0620 23:07:37.739290 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:37.739334 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:37.739717 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:37.739924 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:37.739927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:37.740477 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:37.740641 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.740645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.741276 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.741277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.742072 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.742074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.742393 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.742393 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.742480 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.742850 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:37.742857 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:37.744184 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:37.745828 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:37.745833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:37.745834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:37.746024 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:37.746185 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:37.746186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:37.746186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:37.746275 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:37.746531 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:37.746536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:37.746663 (+ 127us) client_negotiation.cc:769] Sending connection context
0620 23:07:37.746764 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 23:07:37.746901 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260620 23:07:37.747558 4253 heartbeater.cc:347] Connected to a master server at 127.17.188.126:35299
I20260620 23:07:37.747627 4253 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:37.747793 4253 heartbeater.cc:511] Master 127.17.188.126:35299 requested a full tablet report, sending...
I20260620 23:07:37.748140 3794 ts_manager.cc:194] Registered new tserver with Master: 6a7b2c5e67af44a7b0aa4c4dc8d9d84e (127.17.188.67:40985)
I20260620 23:07:37.748456 3794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:41639
I20260620 23:07:37.755842 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:07:37.762571 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.757259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60288 (local address 127.17.188.126:35299)
0620 23:07:37.757308 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.757310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.757412 (+ 102us) server_negotiation.cc:407] Connection header received
0620 23:07:37.757452 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.757453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.757460 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.757515 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:37.758005 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.758462 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.759114 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.759283 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.759331 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.760445 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:37.760446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:37.760447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:37.760452 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:37.761617 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.761950 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.761951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.761951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.761997 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:37.762170 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:37.762171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:37.762171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:37.762278 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:37.762352 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.762471 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.762516 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:07:37.763935 3794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:07:37.765439 3794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:07:37.771533 4261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.767407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:40349 (local address 127.0.0.1:40590)
0620 23:07:37.767570 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:37.767603 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:37.767662 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:37.768077 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:37.768077 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:37.768079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:37.768111 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.768112 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.769426 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.769427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.770504 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.770506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.771203 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.771204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.771306 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.771318 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:07:37.771414 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 23:07:37.771464 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":6,"thread_start_us":56,"threads_started":1}
I20260620 23:07:37.771543 4263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.767501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.17.188.66:40349)
0620 23:07:37.767972 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.767976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.767983 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:07:37.768011 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.768012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.768023 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.768075 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:37.768243 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.769340 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.770590 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.771104 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.771199 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.771275 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.771431 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.771471 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":77,"threads_started":1}
I20260620 23:07:37.771746 4264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.767582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:40985 (local address 127.0.0.1:58438)
0620 23:07:37.767827 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:37.767835 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:37.767875 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:37.768033 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:37.768036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:37.768038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:37.768108 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.768110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.769187 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.769188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.770504 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.770506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.771470 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.771471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.771541 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.771545 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:07:37.771592 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 23:07:37.771639 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":7,"thread_start_us":38,"threads_started":1}
I20260620 23:07:37.771953 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.767708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58438 (local address 127.17.188.67:40985)
0620 23:07:37.767853 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.767856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.767867 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:07:37.767919 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.767920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.767930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.768008 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:37.768263 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.769076 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.770608 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.771312 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.771381 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.771434 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.771852 (+ 418us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.771889 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260620 23:07:37.772123 4262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.767463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45951 (local address 127.0.0.1:54248)
0620 23:07:37.767646 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:37.767655 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:37.767692 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:37.768455 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:37.768456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:37.768458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:37.768512 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.768514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.769692 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.769693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.770599 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.770602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.771506 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.771507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.771557 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.771560 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:37.771758 (+ 198us) client_negotiation.cc:240] Negotiation successful
0620 23:07:37.772077 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":47,"threads_started":1}
I20260620 23:07:37.772225 4265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.767569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.17.188.65:45951)
0620 23:07:37.768244 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.768249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.768259 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:07:37.768294 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.768296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.768310 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.768380 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:37.768580 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.769600 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.770668 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.771374 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.771430 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.771479 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.771875 (+ 396us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.771920 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":76,"threads_started":1}
I20260620 23:07:37.772251 4053 tablet_service.cc:1467] Processing CreateTablet for tablet 049a29f47b8e40a7845cc82f9f5b9c85 (DEFAULT_TABLE table=test-table [id=6b4dbf1efbdb4e938cd5c8f51778336b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:37.772619 4053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049a29f47b8e40a7845cc82f9f5b9c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:37.772823 3917 tablet_service.cc:1467] Processing CreateTablet for tablet 049a29f47b8e40a7845cc82f9f5b9c85 (DEFAULT_TABLE table=test-table [id=6b4dbf1efbdb4e938cd5c8f51778336b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:37.773103 3917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049a29f47b8e40a7845cc82f9f5b9c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:37.773957 4187 tablet_service.cc:1467] Processing CreateTablet for tablet 049a29f47b8e40a7845cc82f9f5b9c85 (DEFAULT_TABLE table=test-table [id=6b4dbf1efbdb4e938cd5c8f51778336b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:37.774253 4187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049a29f47b8e40a7845cc82f9f5b9c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:37.775696 4267 tablet_bootstrap.cc:492] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc: Bootstrap starting.
I20260620 23:07:37.776136 4268 tablet_bootstrap.cc:492] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: Bootstrap starting.
I20260620 23:07:37.776438 4267 tablet_bootstrap.cc:654] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:37.776737 4268 tablet_bootstrap.cc:654] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:37.776947 4267 log.cc:824] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:37.777287 4268 log.cc:824] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:37.777716 4267 tablet_bootstrap.cc:492] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc: No bootstrap required, opened a new log
I20260620 23:07:37.777689 4270 tablet_bootstrap.cc:492] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Bootstrap starting.
I20260620 23:07:37.777784 4267 ts_tablet_manager.cc:1397] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:37.778035 4268 tablet_bootstrap.cc:492] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: No bootstrap required, opened a new log
I20260620 23:07:37.778102 4268 ts_tablet_manager.cc:1397] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:07:37.778296 4270 tablet_bootstrap.cc:654] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:37.778821 4270 log.cc:824] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:37.779482 4270 tablet_bootstrap.cc:492] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: No bootstrap required, opened a new log
I20260620 23:07:37.779547 4270 ts_tablet_manager.cc:1397] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:37.779440 4267 raft_consensus.cc:348] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } }
I20260620 23:07:37.779614 4267 raft_consensus.cc:374] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:37.779654 4267 raft_consensus.cc:729] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac337f97014c3aa6ccc655f21399dc, State: Initialized, Role: FOLLOWER
I20260620 23:07:37.779770 4267 consensus_queue.cc:260] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } }
I20260620 23:07:37.780066 4267 ts_tablet_manager.cc:1428] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:07:37.780086 3985 heartbeater.cc:503] Master 127.17.188.126:35299 was elected leader, sending a full tablet report...
I20260620 23:07:37.780308 4268 raft_consensus.cc:348] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } }
I20260620 23:07:37.780470 4268 raft_consensus.cc:374] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:37.780512 4268 raft_consensus.cc:729] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2261d86593249d1907ddb16350d507c, State: Initialized, Role: FOLLOWER
I20260620 23:07:37.780645 4268 consensus_queue.cc:260] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } }
I20260620 23:07:37.780917 4268 ts_tablet_manager.cc:1428] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 23:07:37.781025 4119 heartbeater.cc:503] Master 127.17.188.126:35299 was elected leader, sending a full tablet report...
I20260620 23:07:37.781523 4270 raft_consensus.cc:348] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } }
I20260620 23:07:37.781698 4270 raft_consensus.cc:374] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:37.781739 4270 raft_consensus.cc:729] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a7b2c5e67af44a7b0aa4c4dc8d9d84e, State: Initialized, Role: FOLLOWER
I20260620 23:07:37.781857 4270 consensus_queue.cc:260] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } }
I20260620 23:07:37.782121 4270 ts_tablet_manager.cc:1428] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:07:37.782155 4253 heartbeater.cc:503] Master 127.17.188.126:35299 was elected leader, sending a full tablet report...
W20260620 23:07:37.853497 4120 tablet.cc:2369] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:07:37.902420 4275 raft_consensus.cc:482] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:37.902555 4275 raft_consensus.cc:504] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } }
I20260620 23:07:37.902953 4275 leader_election.cc:288] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2261d86593249d1907ddb16350d507c (127.17.188.66:40349), 66ac337f97014c3aa6ccc655f21399dc (127.17.188.65:45951)
I20260620 23:07:37.906466 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.903123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:40349 (local address 127.17.188.67:45851)
0620 23:07:37.903176 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:37.903188 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:37.903248 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:37.903439 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:37.903439 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:37.903441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:37.903526 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.903528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.904190 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.904191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.905446 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.905449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.906222 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.906224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.906319 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.906322 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:37.906365 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 23:07:37.906399 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":12}
I20260620 23:07:37.906495 4265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.903297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:45911 (local address 127.17.188.65:45951)
0620 23:07:37.903339 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.903341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.903420 (+ 79us) server_negotiation.cc:407] Connection header received
0620 23:07:37.903448 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.903449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.903461 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.903521 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:37.903693 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.904134 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.905570 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.906128 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.906175 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.906183 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.906389 (+ 206us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.906428 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:07:37.906646 4263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.903187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:45851 (local address 127.17.188.66:40349)
0620 23:07:37.903234 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.903236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.903246 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:07:37.903352 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.903353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.903363 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.903422 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:37.903610 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.904113 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.905558 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.906117 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.906184 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.906192 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.906553 (+ 361us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.906591 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:07:37.906842 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.903170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45951 (local address 127.17.188.67:45911)
0620 23:07:37.903323 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:37.903334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:37.903399 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:37.903532 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:37.903533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:37.903534 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:37.903618 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.903620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.904238 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.904238 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.905452 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:37.905455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.906237 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.906238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:37.906315 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.906319 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:07:37.906726 (+ 407us) client_negotiation.cc:240] Negotiation successful
0620 23:07:37.906777 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
I20260620 23:07:37.911257 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049a29f47b8e40a7845cc82f9f5b9c85" candidate_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac337f97014c3aa6ccc655f21399dc" is_pre_election: true
I20260620 23:07:37.911422 3936 raft_consensus.cc:2454] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a7b2c5e67af44a7b0aa4c4dc8d9d84e in term 0.
I20260620 23:07:37.911398 4073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049a29f47b8e40a7845cc82f9f5b9c85" candidate_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2261d86593249d1907ddb16350d507c" is_pre_election: true
I20260620 23:07:37.911561 4073 raft_consensus.cc:2454] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a7b2c5e67af44a7b0aa4c4dc8d9d84e in term 0.
I20260620 23:07:37.911651 4141 leader_election.cc:302] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac337f97014c3aa6ccc655f21399dc, 6a7b2c5e67af44a7b0aa4c4dc8d9d84e; no voters:
I20260620 23:07:37.911777 4275 raft_consensus.cc:2781] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:07:37.911818 4275 raft_consensus.cc:482] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:07:37.911837 4275 raft_consensus.cc:3037] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:37.912699 4275 raft_consensus.cc:504] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } }
I20260620 23:07:37.912870 4275 leader_election.cc:288] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [CANDIDATE]: Term 1 election: Requested vote from peers c2261d86593249d1907ddb16350d507c (127.17.188.66:40349), 66ac337f97014c3aa6ccc655f21399dc (127.17.188.65:45951)
I20260620 23:07:37.913048 4073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049a29f47b8e40a7845cc82f9f5b9c85" candidate_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2261d86593249d1907ddb16350d507c"
I20260620 23:07:37.913049 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049a29f47b8e40a7845cc82f9f5b9c85" candidate_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac337f97014c3aa6ccc655f21399dc"
I20260620 23:07:37.913132 4073 raft_consensus.cc:3037] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:37.913133 3936 raft_consensus.cc:3037] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:37.914050 4073 raft_consensus.cc:2454] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a7b2c5e67af44a7b0aa4c4dc8d9d84e in term 1.
I20260620 23:07:37.914052 3936 raft_consensus.cc:2454] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a7b2c5e67af44a7b0aa4c4dc8d9d84e in term 1.
I20260620 23:07:37.914228 4141 leader_election.cc:302] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac337f97014c3aa6ccc655f21399dc, 6a7b2c5e67af44a7b0aa4c4dc8d9d84e; no voters:
I20260620 23:07:37.914353 4275 raft_consensus.cc:2781] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:37.914585 4275 raft_consensus.cc:686] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 1 LEADER]: Becoming Leader. State: Replica: 6a7b2c5e67af44a7b0aa4c4dc8d9d84e, State: Running, Role: LEADER
I20260620 23:07:37.914691 4275 consensus_queue.cc:237] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } }
I20260620 23:07:37.915562 3794 catalog_manager.cc:5495] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a7b2c5e67af44a7b0aa4c4dc8d9d84e (127.17.188.67). New cstate: current_term: 1 leader_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a7b2c5e67af44a7b0aa4c4dc8d9d84e" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 } health_report { overall_health: UNKNOWN } } }
I20260620 23:07:37.954381 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.952362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58450 (local address 127.17.188.67:40985)
0620 23:07:37.952417 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:37.952419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:37.952427 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:07:37.952466 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:37.952467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:37.952482 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:37.952541 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:07:37.952664 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.953094 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.953780 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:37.953948 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:37.953995 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:37.954118 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:07:37.954195 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:07:37.954269 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:37.954307 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 23:07:37.954316 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:07:37.956071 4073 raft_consensus.cc:1261] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [term 1 FOLLOWER]: Refusing update from remote peer 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:37.956071 3936 raft_consensus.cc:1261] T 049a29f47b8e40a7845cc82f9f5b9c85 P 66ac337f97014c3aa6ccc655f21399dc [term 1 FOLLOWER]: Refusing update from remote peer 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:37.956424 4277 consensus_queue.cc:1035] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac337f97014c3aa6ccc655f21399dc" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 45951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:37.956563 4275 consensus_queue.cc:1035] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2261d86593249d1907ddb16350d507c" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 40349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:37.959003 4282 mvcc.cc:205] Tried to move back new op lower bound from 7299059130185523200 to 7299059130019385344. Current Snapshot: MvccSnapshot[applied={T|T < 7299059130185523200}]
I20260620 23:07:37.959403 4280 mvcc.cc:205] Tried to move back new op lower bound from 7299059130185523200 to 7299059130019385344. Current Snapshot: MvccSnapshot[applied={T|T < 7299059130185523200}]
I20260620 23:07:37.959834 4281 mvcc.cc:205] Tried to move back new op lower bound from 7299059130185523200 to 7299059130019385344. Current Snapshot: MvccSnapshot[applied={T|T < 7299059130185523200 or (T in {7299059130185523200})}]
I20260620 23:07:37.961426 3794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:37.961522 3794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:37.963079 3794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1861d43c6e334ad99bbd1dde1e290a61: Sending DeleteTablet for 3 replicas of tablet 049a29f47b8e40a7845cc82f9f5b9c85
I20260620 23:07:37.963454 4053 tablet_service.cc:1514] Processing DeleteTablet for tablet 049a29f47b8e40a7845cc82f9f5b9c85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:37 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:40590
I20260620 23:07:37.963845 4187 tablet_service.cc:1514] Processing DeleteTablet for tablet 049a29f47b8e40a7845cc82f9f5b9c85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:37 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:58438
I20260620 23:07:37.964169 4290 tablet_replica.cc:330] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: stopping tablet replica
I20260620 23:07:37.964334 4290 raft_consensus.cc:2229] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:37.964509 4290 raft_consensus.cc:2258] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:37.965080 4290 ts_tablet_manager.cc:1905] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:37.965545 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 049a29f47b8e40a7845cc82f9f5b9c85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:37 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:54248
I20260620 23:07:37.965929 4291 tablet_replica.cc:330] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: stopping tablet replica
I20260620 23:07:37.966035 4291 raft_consensus.cc:2229] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:07:37.966142 4291 raft_consensus.cc:2258] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:37.966555 4291 ts_tablet_manager.cc:1905] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:37.966233 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 3856
I20260620 23:07:37.968600 4291 ts_tablet_manager.cc:1918] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:37.968673 4291 log.cc:1196] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/049a29f47b8e40a7845cc82f9f5b9c85
I20260620 23:07:37.968982 4291 ts_tablet_manager.cc:1939] T 049a29f47b8e40a7845cc82f9f5b9c85 P 6a7b2c5e67af44a7b0aa4c4dc8d9d84e: Deleting consensus metadata
I20260620 23:07:37.969394 4290 ts_tablet_manager.cc:1918] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:37.969496 4290 log.cc:1196] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/049a29f47b8e40a7845cc82f9f5b9c85
I20260620 23:07:37.969769 4290 ts_tablet_manager.cc:1939] T 049a29f47b8e40a7845cc82f9f5b9c85 P c2261d86593249d1907ddb16350d507c: Deleting consensus metadata
I20260620 23:07:37.972436 3781 catalog_manager.cc:4841] TS c2261d86593249d1907ddb16350d507c (127.17.188.66:40349): tablet 049a29f47b8e40a7845cc82f9f5b9c85 (table test-table [id=6b4dbf1efbdb4e938cd5c8f51778336b]) successfully deleted
I20260620 23:07:37.972535 3781 catalog_manager.cc:4841] TS 6a7b2c5e67af44a7b0aa4c4dc8d9d84e (127.17.188.67:40985): tablet 049a29f47b8e40a7845cc82f9f5b9c85 (table test-table [id=6b4dbf1efbdb4e938cd5c8f51778336b]) successfully deleted
W20260620 23:07:37.976518 3779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:45951 (error 108)
I20260620 23:07:37.976991 4262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:37.976861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:45951 (local address 127.0.0.1:54260)
0620 23:07:37.976913 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:37.976949 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 23:07:37.977030 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 3990
W20260620 23:07:37.979683 3779 catalog_manager.cc:4584] TS 66ac337f97014c3aa6ccc655f21399dc (127.17.188.65:45951): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 049a29f47b8e40a7845cc82f9f5b9c85: Network error: Client connection negotiation failed: client connection to 127.17.188.65:45951: connect: Connection refused (error 111)
I20260620 23:07:37.982767 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 4124
I20260620 23:07:37.987541 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 3762
2026-06-20T23:07:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1965 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T23:07:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:38Z Disabled control of system clock
I20260620 23:07:38.014252 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:36943
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:36943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:07:38.092475 4297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:38.092665 4297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:38.092684 4297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:38.093815 4297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:38.093853 4297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:38.093866 4297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:38.093878 4297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:38.093930 4297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:38.095237 4297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:36943
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:36943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:38.095479 4297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:38.095772 4297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:38.100374 4305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:38.100461 4303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:38.100553 4297 server_base.cc:1034] running on GCE node
W20260620 23:07:38.100474 4302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:38.100843 4297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:38.101128 4297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:38.102270 4297 hybrid_clock.cc:648] HybridClock initialized: now 1781996858102237 us; error 45 us; skew 500 ppm
I20260620 23:07:38.107950 4297 webserver.cc:460] Webserver started at http://127.17.188.126:45819/ using document root <none> and password file <none>
I20260620 23:07:38.108207 4297 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:38.108254 4297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:38.108394 4297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:38.109503 4297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "f3abe11b3a774eddb275c4914a979aef"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "6a2e42deb39d641096ae442f4b691199"
server_key_iv: "2e8d909f27779f5676c751547d63f2e4"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.109849 4297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "f3abe11b3a774eddb275c4914a979aef"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "6a2e42deb39d641096ae442f4b691199"
server_key_iv: "2e8d909f27779f5676c751547d63f2e4"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.111089 4297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:38.111785 4311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.111967 4297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:38.112051 4297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "f3abe11b3a774eddb275c4914a979aef"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "6a2e42deb39d641096ae442f4b691199"
server_key_iv: "2e8d909f27779f5676c751547d63f2e4"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.112115 4297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:38.128012 4297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:38.128288 4297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:38.128388 4297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:38.131966 4297 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:36943
I20260620 23:07:38.132012 4363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:36943 every 8 connection(s)
I20260620 23:07:38.132284 4297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:38.133109 4364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:38.135164 4364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef: Bootstrap starting.
I20260620 23:07:38.135691 4364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:38.135973 4364 log.cc:824] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:38.136569 4364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef: No bootstrap required, opened a new log
I20260620 23:07:38.138103 4364 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3abe11b3a774eddb275c4914a979aef" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36943 } }
I20260620 23:07:38.138226 4364 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:38.138247 4364 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3abe11b3a774eddb275c4914a979aef, State: Initialized, Role: FOLLOWER
I20260620 23:07:38.138417 4364 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3abe11b3a774eddb275c4914a979aef" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36943 } }
I20260620 23:07:38.138476 4364 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:38.138494 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 4297
I20260620 23:07:38.138525 4364 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:38.138564 4364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:38.138604 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:38.138782 18161 external_mini_cluster.cc:1428] Setting key 400468f499b74e3abc846e0561433bb3
I20260620 23:07:38.139169 4364 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3abe11b3a774eddb275c4914a979aef" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36943 } }
I20260620 23:07:38.139282 4364 leader_election.cc:302] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3abe11b3a774eddb275c4914a979aef; no voters:
I20260620 23:07:38.139442 4364 leader_election.cc:288] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:38.139482 4369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:38.139611 4369 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [term 1 LEADER]: Becoming Leader. State: Replica: f3abe11b3a774eddb275c4914a979aef, State: Running, Role: LEADER
I20260620 23:07:38.139679 4364 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:38.139693 4369 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3abe11b3a774eddb275c4914a979aef" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36943 } }
I20260620 23:07:38.140163 4369 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3abe11b3a774eddb275c4914a979aef. Latest consensus state: current_term: 1 leader_uuid: "f3abe11b3a774eddb275c4914a979aef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3abe11b3a774eddb275c4914a979aef" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36943 } } }
I20260620 23:07:38.140239 4369 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:38.140420 4370 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3abe11b3a774eddb275c4914a979aef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3abe11b3a774eddb275c4914a979aef" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 36943 } } }
I20260620 23:07:38.140504 4370 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:38.140606 4377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:38.141067 4377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:38.142546 4377 catalog_manager.cc:1347] Generated new cluster ID: 3213c77979294122b19e01124fab7364
I20260620 23:07:38.142596 4377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:38.144698 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.139234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.17.188.126:36943)
0620 23:07:38.139478 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.139487 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.139519 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:07:38.139662 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.139669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.139738 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.140119 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:38.140210 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.141900 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.142840 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.143225 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.143346 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.144013 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:38.144016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:38.144017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:38.144193 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:38.144223 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:38.144233 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:38.144310 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.144387 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.144547 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260620 23:07:38.162487 4377 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:38.163205 4377 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:38.167694 4377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef: Generated new TSK 0
I20260620 23:07:38.167855 4377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:38.173650 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36943
--builtin_ntp_servers=127.17.188.84:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:07:38.253021 4388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:38.253197 4388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:38.253217 4388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:38.254381 4388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:38.254421 4388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:38.254446 4388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:07:38.255756 4388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:38.255971 4388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:38.256256 4388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:38.261724 4394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:38.261831 4388 server_base.cc:1034] running on GCE node
W20260620 23:07:38.261770 4396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:38.261873 4393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:38.262159 4388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:38.262426 4388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:38.263584 4388 hybrid_clock.cc:648] HybridClock initialized: now 1781996858263573 us; error 28 us; skew 500 ppm
I20260620 23:07:38.264770 4388 webserver.cc:460] Webserver started at http://127.17.188.65:38807/ using document root <none> and password file <none>
I20260620 23:07:38.265008 4388 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:38.265075 4388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:38.265202 4388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:38.266189 4388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "25e8908db17f4eb0b6ee49b49deb5694"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "0d1dfee9d5d5c7f58aabc7a67b4ae426"
server_key_iv: "c33708db595abdf6a42b89a6deb82308"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.266556 4388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "25e8908db17f4eb0b6ee49b49deb5694"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "0d1dfee9d5d5c7f58aabc7a67b4ae426"
server_key_iv: "c33708db595abdf6a42b89a6deb82308"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.267779 4388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 23:07:38.268446 4402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.268597 4388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:38.268693 4388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "25e8908db17f4eb0b6ee49b49deb5694"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "0d1dfee9d5d5c7f58aabc7a67b4ae426"
server_key_iv: "c33708db595abdf6a42b89a6deb82308"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.268764 4388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:38.275991 4388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:38.276270 4388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:38.276405 4388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:38.276597 4388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:38.277062 4388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:38.277130 4388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.277177 4388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:38.277204 4388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.282658 4388 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:33237
I20260620 23:07:38.282697 4515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:33237 every 8 connection(s)
I20260620 23:07:38.283034 4388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:07:38.287628 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.283677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:38061 (local address 127.17.188.126:36943)
0620 23:07:38.283747 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.283749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.283964 (+ 215us) server_negotiation.cc:407] Connection header received
0620 23:07:38.284378 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.284379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.284391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.284446 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:38.285061 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.285614 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.286473 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.286657 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.286706 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.287406 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:38.287407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:38.287407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:38.287411 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:38.287424 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:38.287427 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:38.287468 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.287542 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.287574 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:07:38.287794 4518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.283662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36943 (local address 127.17.188.65:38061)
0620 23:07:38.283837 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:38.283873 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:38.284301 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:38.284490 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:38.284493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:38.284500 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:38.284972 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.284975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.285686 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.285687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.286401 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.286403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.286721 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.286722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.286833 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.287286 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:07:38.287286 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:38.287310 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:07:38.287330 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:38.287477 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:38.287480 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:38.287482 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:07:38.287529 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 23:07:38.287595 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260620 23:07:38.288295 4516 heartbeater.cc:347] Connected to a master server at 127.17.188.126:36943
I20260620 23:07:38.288389 4516 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:38.288441 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 4388
I20260620 23:07:38.288538 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:07:38.288602 4516 heartbeater.cc:511] Master 127.17.188.126:36943 requested a full tablet report, sending...
I20260620 23:07:38.288745 18161 external_mini_cluster.cc:1428] Setting key 2737d4c3ffffeddfa081ed8c5160ce0c
I20260620 23:07:38.289100 4328 ts_manager.cc:194] Registered new tserver with Master: 25e8908db17f4eb0b6ee49b49deb5694 (127.17.188.65:33237)
I20260620 23:07:38.289794 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36943
--builtin_ntp_servers=127.17.188.84:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:07:38.289978 4328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.65:38061
W20260620 23:07:38.366415 4519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:38.366593 4519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:38.366613 4519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:38.367671 4519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:38.367705 4519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:38.367726 4519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:07:38.368961 4519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:38.369174 4519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:38.369462 4519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:38.374396 4525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:38.374384 4527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:38.374389 4524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:38.374475 4519 server_base.cc:1034] running on GCE node
I20260620 23:07:38.374858 4519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:38.375152 4519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:38.376323 4519 hybrid_clock.cc:648] HybridClock initialized: now 1781996858376311 us; error 24 us; skew 500 ppm
I20260620 23:07:38.377537 4519 webserver.cc:460] Webserver started at http://127.17.188.66:42377/ using document root <none> and password file <none>
I20260620 23:07:38.377796 4519 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:38.377878 4519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:38.378024 4519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:38.379112 4519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "b752fcd9b59a45d7be3e801be6365ecf"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "fa534e52cb7d1ecc299e66a59a35a6c7"
server_key_iv: "2c6900498c350153eb9cb72781e54ef1"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.379479 4519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "b752fcd9b59a45d7be3e801be6365ecf"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "fa534e52cb7d1ecc299e66a59a35a6c7"
server_key_iv: "2c6900498c350153eb9cb72781e54ef1"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.380771 4519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:07:38.381541 4533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.381708 4519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:38.381808 4519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "b752fcd9b59a45d7be3e801be6365ecf"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "fa534e52cb7d1ecc299e66a59a35a6c7"
server_key_iv: "2c6900498c350153eb9cb72781e54ef1"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.381892 4519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:38.396675 4519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:38.396945 4519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:38.397100 4519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:38.397365 4519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:38.397673 4519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:38.397729 4519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.397765 4519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:38.397794 4519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.403728 4519 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:42343
I20260620 23:07:38.403795 4646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:42343 every 8 connection(s)
I20260620 23:07:38.404170 4519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:07:38.404368 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 4519
I20260620 23:07:38.404481 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:07:38.404760 18161 external_mini_cluster.cc:1428] Setting key d0796478e15734e603b44c8fb01f8ced
I20260620 23:07:38.405815 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36943
--builtin_ntp_servers=127.17.188.84:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:07:38.409955 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.404981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:57551 (local address 127.17.188.126:36943)
0620 23:07:38.405051 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.405053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.405361 (+ 308us) server_negotiation.cc:407] Connection header received
0620 23:07:38.405866 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.405867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.405878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.405927 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:38.406549 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.407040 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.408536 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.408743 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.408817 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.409665 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:38.409666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:38.409666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:38.409671 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:38.409683 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:38.409687 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:38.409753 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.409855 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.409880 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:07:38.410187 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.404969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36943 (local address 127.17.188.66:57551)
0620 23:07:38.405187 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:38.405226 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:38.405767 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:38.405997 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:38.406001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:38.406010 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:38.406463 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.406467 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.407131 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.407132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.408460 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.408462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.408824 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.408825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.408956 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.409527 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:07:38.409527 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:38.409552 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:07:38.409573 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:38.409753 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:38.409769 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:38.409772 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:38.409856 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 23:07:38.409949 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260620 23:07:38.410674 4647 heartbeater.cc:347] Connected to a master server at 127.17.188.126:36943
I20260620 23:07:38.410796 4647 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:38.410971 4647 heartbeater.cc:511] Master 127.17.188.126:36943 requested a full tablet report, sending...
I20260620 23:07:38.411334 4328 ts_manager.cc:194] Registered new tserver with Master: b752fcd9b59a45d7be3e801be6365ecf (127.17.188.66:42343)
I20260620 23:07:38.411654 4328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.66:57551
W20260620 23:07:38.488070 4650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:38.488292 4650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:38.488330 4650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:38.489513 4650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:38.489567 4650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:38.489629 4650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:07:38.490967 4650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:36943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:38.491225 4650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:38.491535 4650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:38.496490 4658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:38.496506 4656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:38.496623 4650 server_base.cc:1034] running on GCE node
W20260620 23:07:38.496505 4655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:38.496879 4650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:38.497146 4650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:38.498296 4650 hybrid_clock.cc:648] HybridClock initialized: now 1781996858498287 us; error 56 us; skew 500 ppm
I20260620 23:07:38.499523 4650 webserver.cc:460] Webserver started at http://127.17.188.67:40231/ using document root <none> and password file <none>
I20260620 23:07:38.499789 4650 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:38.499858 4650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:38.500010 4650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:38.500988 4650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "fdc3a2c7ff534e9db7ce9785af743d9a"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "7ecb93b64864463c646a56e487b170b3"
server_key_iv: "baf7993dfd0dd0504da55589fd73e57b"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.501389 4650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "fdc3a2c7ff534e9db7ce9785af743d9a"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "7ecb93b64864463c646a56e487b170b3"
server_key_iv: "baf7993dfd0dd0504da55589fd73e57b"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.502631 4650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:07:38.503320 4664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.503492 4650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:38.503584 4650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "fdc3a2c7ff534e9db7ce9785af743d9a"
format_stamp: "Formatted at 2026-06-20 23:07:38 on dist-test-slave-zpss"
server_key: "7ecb93b64864463c646a56e487b170b3"
server_key_iv: "baf7993dfd0dd0504da55589fd73e57b"
server_key_version: "encryptionkey@0"
I20260620 23:07:38.503667 4650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:38.530982 4650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:38.531262 4650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:38.531406 4650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:38.531615 4650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:38.531927 4650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:38.531980 4650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.532019 4650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:38.532047 4650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:38.537575 4650 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:45279
I20260620 23:07:38.537657 4777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:45279 every 8 connection(s)
I20260620 23:07:38.537942 4650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:07:38.540342 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 4650
I20260620 23:07:38.540457 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:07:38.540645 18161 external_mini_cluster.cc:1428] Setting key 54e1b99c624e6c164e407ccead9b5a99
I20260620 23:07:38.543588 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.538656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:56457 (local address 127.17.188.126:36943)
0620 23:07:38.538747 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.538749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.538948 (+ 199us) server_negotiation.cc:407] Connection header received
0620 23:07:38.539566 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.539568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.539583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.539646 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:38.540436 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.540940 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.542102 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.542306 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.542391 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.543274 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:38.543276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:38.543276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:38.543281 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:38.543296 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:38.543300 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:38.543384 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.543472 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.543501 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:07:38.543790 4780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.538618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:36943 (local address 127.17.188.67:56457)
0620 23:07:38.538822 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:38.538861 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:38.539436 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:38.539713 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:38.539717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:38.539726 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:38.540326 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.540332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.541061 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.541062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.541977 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.541980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.542398 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.542399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.542545 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.543113 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:07:38.543114 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:38.543141 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:07:38.543163 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:38.543387 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:38.543393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:38.543396 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:38.543474 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 23:07:38.543570 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
I20260620 23:07:38.544253 4778 heartbeater.cc:347] Connected to a master server at 127.17.188.126:36943
I20260620 23:07:38.544325 4778 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:38.544512 4778 heartbeater.cc:511] Master 127.17.188.126:36943 requested a full tablet report, sending...
I20260620 23:07:38.544862 4328 ts_manager.cc:194] Registered new tserver with Master: fdc3a2c7ff534e9db7ce9785af743d9a (127.17.188.67:45279)
I20260620 23:07:38.545207 4328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.67:56457
I20260620 23:07:38.552176 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:07:38.556068 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.553543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.17.188.126:36943)
0620 23:07:38.553610 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.553612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.553710 (+ 98us) server_negotiation.cc:407] Connection header received
0620 23:07:38.553743 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.553744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.553752 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.553805 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:38.553950 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.554355 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.555029 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.555185 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.555229 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.555839 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:38.555840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:38.555841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:38.555844 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:38.555852 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:38.555854 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:38.555904 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.555975 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.555985 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 23:07:38.557487 4328 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:07:38.558889 4328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:07:38.564672 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.560692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:42343 (local address 127.0.0.1:36248)
0620 23:07:38.560900 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:38.560927 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:38.560986 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:38.561406 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:38.561407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:38.561409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:38.561444 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.561446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.562735 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.562736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.563487 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.563490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.564364 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.564365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.564443 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.564456 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 23:07:38.564532 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:07:38.564601 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":3,"thread_start_us":72,"threads_started":1}
I20260620 23:07:38.564673 4790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.560992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.17.188.66:42343)
0620 23:07:38.561167 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.561171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.561181 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:07:38.561216 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.561218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.561245 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.561316 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:38.561524 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.562664 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.563598 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.564272 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.564342 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.564418 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.564554 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.564591 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260620 23:07:38.564845 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.560646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:33237 (local address 127.0.0.1:38354)
0620 23:07:38.560821 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:38.560856 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:38.560918 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:38.561153 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:38.561155 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:38.561157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:38.561218 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.561230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.562514 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.562516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.563817 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.563820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.564630 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.564631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.564688 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.564693 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:07:38.564723 (+ 30us) client_negotiation.cc:240] Negotiation successful
0620 23:07:38.564774 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":53,"thread_start_us":65,"threads_started":1}
I20260620 23:07:38.565218 4788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.560744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.17.188.65:33237)
0620 23:07:38.560881 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.560885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.560900 (+ 15us) server_negotiation.cc:407] Connection header received
0620 23:07:38.561038 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.561041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.561058 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.561133 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:38.561330 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.562420 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.563904 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.564543 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.564971 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.565068 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.565093 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.565128 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260620 23:07:38.565229 4789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.560838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:45279 (local address 127.0.0.1:52730)
0620 23:07:38.560967 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:38.560976 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:38.561016 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:38.561546 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:38.561547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:38.561554 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:38.561603 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.561604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.562511 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.562512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.564002 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.564004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.565022 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.565024 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.565087 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.565090 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:38.565119 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 23:07:38.565157 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":36,"thread_start_us":86,"threads_started":1}
I20260620 23:07:38.565577 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.561207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.17.188.67:45279)
0620 23:07:38.561389 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.561393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.561402 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:07:38.561429 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.561431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.561443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.561543 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:38.561659 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.562416 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.564079 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.564921 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.565363 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.565454 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.565478 (+ 24us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.565508 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":124,"threads_started":1}
I20260620 23:07:38.565604 4581 tablet_service.cc:1467] Processing CreateTablet for tablet 3685c79c9a404333ae0bce02523eab4b (DEFAULT_TABLE table=test-table [id=9dd616c0e2fc443d953a0524ca7d7faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:38.565754 4450 tablet_service.cc:1467] Processing CreateTablet for tablet 3685c79c9a404333ae0bce02523eab4b (DEFAULT_TABLE table=test-table [id=9dd616c0e2fc443d953a0524ca7d7faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:38.565959 4581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3685c79c9a404333ae0bce02523eab4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:38.566023 4450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3685c79c9a404333ae0bce02523eab4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:38.566135 4712 tablet_service.cc:1467] Processing CreateTablet for tablet 3685c79c9a404333ae0bce02523eab4b (DEFAULT_TABLE table=test-table [id=9dd616c0e2fc443d953a0524ca7d7faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:38.566339 4712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3685c79c9a404333ae0bce02523eab4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:38.568635 4792 tablet_bootstrap.cc:492] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: Bootstrap starting.
I20260620 23:07:38.569103 4793 tablet_bootstrap.cc:492] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: Bootstrap starting.
I20260620 23:07:38.569453 4794 tablet_bootstrap.cc:492] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694: Bootstrap starting.
I20260620 23:07:38.569556 4792 tablet_bootstrap.cc:654] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:38.569773 4793 tablet_bootstrap.cc:654] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:38.569864 4792 log.cc:824] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:38.570093 4793 log.cc:824] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:38.570118 4794 tablet_bootstrap.cc:654] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:38.570430 4794 log.cc:824] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:38.570736 4792 tablet_bootstrap.cc:492] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: No bootstrap required, opened a new log
I20260620 23:07:38.570818 4792 ts_tablet_manager.cc:1397] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:38.571138 4794 tablet_bootstrap.cc:492] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694: No bootstrap required, opened a new log
I20260620 23:07:38.571189 4793 tablet_bootstrap.cc:492] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: No bootstrap required, opened a new log
I20260620 23:07:38.571259 4794 ts_tablet_manager.cc:1397] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:07:38.571252 4793 ts_tablet_manager.cc:1397] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:07:38.573217 4794 raft_consensus.cc:348] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } }
I20260620 23:07:38.573396 4794 raft_consensus.cc:374] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:38.573441 4794 raft_consensus.cc:729] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e8908db17f4eb0b6ee49b49deb5694, State: Initialized, Role: FOLLOWER
I20260620 23:07:38.573539 4794 consensus_queue.cc:260] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } }
I20260620 23:07:38.573531 4792 raft_consensus.cc:348] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } }
I20260620 23:07:38.573684 4792 raft_consensus.cc:374] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:38.573725 4792 raft_consensus.cc:729] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc3a2c7ff534e9db7ce9785af743d9a, State: Initialized, Role: FOLLOWER
I20260620 23:07:38.573793 4794 ts_tablet_manager.cc:1428] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:07:38.573926 4516 heartbeater.cc:503] Master 127.17.188.126:36943 was elected leader, sending a full tablet report...
I20260620 23:07:38.573902 4792 consensus_queue.cc:260] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } }
I20260620 23:07:38.573870 4793 raft_consensus.cc:348] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } }
I20260620 23:07:38.574008 4793 raft_consensus.cc:374] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:38.574060 4793 raft_consensus.cc:729] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b752fcd9b59a45d7be3e801be6365ecf, State: Initialized, Role: FOLLOWER
I20260620 23:07:38.574141 4792 ts_tablet_manager.cc:1428] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:07:38.574186 4793 consensus_queue.cc:260] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } }
I20260620 23:07:38.574337 4778 heartbeater.cc:503] Master 127.17.188.126:36943 was elected leader, sending a full tablet report...
I20260620 23:07:38.574419 4793 ts_tablet_manager.cc:1428] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 23:07:38.574512 4647 heartbeater.cc:503] Master 127.17.188.126:36943 was elected leader, sending a full tablet report...
I20260620 23:07:38.633687 4800 raft_consensus.cc:482] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:38.633795 4800 raft_consensus.cc:504] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } }
I20260620 23:07:38.634069 4800 leader_election.cc:288] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e8908db17f4eb0b6ee49b49deb5694 (127.17.188.65:33237), fdc3a2c7ff534e9db7ce9785af743d9a (127.17.188.67:45279)
I20260620 23:07:38.636904 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.634252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:33237 (local address 127.17.188.66:46671)
0620 23:07:38.634316 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:38.634329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:38.634369 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:38.634522 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:38.634523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:38.634525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:38.634591 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.634593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.635247 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.635248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.635981 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.635983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.636720 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.636721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.636777 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.636780 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:38.636824 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:07:38.636853 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 23:07:38.636970 4788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.634316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:46671 (local address 127.17.188.65:33237)
0620 23:07:38.634394 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.634396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.634406 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:07:38.634435 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.634436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.634448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.634515 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:38.634688 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.635163 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.636077 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.636634 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.636718 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.636726 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.636850 (+ 124us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.636887 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:07:38.637225 4470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3685c79c9a404333ae0bce02523eab4b" candidate_uuid: "b752fcd9b59a45d7be3e801be6365ecf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e8908db17f4eb0b6ee49b49deb5694" is_pre_election: true
I20260620 23:07:38.637387 4801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.634384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:45279 (local address 127.17.188.66:36291)
0620 23:07:38.634623 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:38.634631 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:38.634697 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:38.634849 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:38.634850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:38.634852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:38.634924 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.634925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.635531 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.635532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.636241 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:38.636243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.637123 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.637125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:38.637193 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.637196 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:38.637254 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 23:07:38.637289 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":131,"threads_started":1}
I20260620 23:07:38.637414 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.634454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:36291 (local address 127.17.188.67:45279)
0620 23:07:38.634519 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.634521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.634693 (+ 172us) server_negotiation.cc:407] Connection header received
0620 23:07:38.634761 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.634762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.634770 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.634831 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:38.634993 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.635442 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.636348 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.636894 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.636960 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.636966 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.637265 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.637303 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:07:38.637418 4470 raft_consensus.cc:2454] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b752fcd9b59a45d7be3e801be6365ecf in term 0.
I20260620 23:07:38.637708 4534 leader_election.cc:302] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e8908db17f4eb0b6ee49b49deb5694, b752fcd9b59a45d7be3e801be6365ecf; no voters:
I20260620 23:07:38.637828 4800 raft_consensus.cc:2781] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:07:38.637814 4732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3685c79c9a404333ae0bce02523eab4b" candidate_uuid: "b752fcd9b59a45d7be3e801be6365ecf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" is_pre_election: true
I20260620 23:07:38.637881 4800 raft_consensus.cc:482] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:07:38.637935 4800 raft_consensus.cc:3037] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:38.637940 4732 raft_consensus.cc:2454] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b752fcd9b59a45d7be3e801be6365ecf in term 0.
I20260620 23:07:38.638630 4800 raft_consensus.cc:504] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } }
I20260620 23:07:38.638746 4800 leader_election.cc:288] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [CANDIDATE]: Term 1 election: Requested vote from peers 25e8908db17f4eb0b6ee49b49deb5694 (127.17.188.65:33237), fdc3a2c7ff534e9db7ce9785af743d9a (127.17.188.67:45279)
I20260620 23:07:38.638936 4470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3685c79c9a404333ae0bce02523eab4b" candidate_uuid: "b752fcd9b59a45d7be3e801be6365ecf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e8908db17f4eb0b6ee49b49deb5694"
I20260620 23:07:38.638960 4732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3685c79c9a404333ae0bce02523eab4b" candidate_uuid: "b752fcd9b59a45d7be3e801be6365ecf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a"
I20260620 23:07:38.639012 4470 raft_consensus.cc:3037] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:38.639020 4732 raft_consensus.cc:3037] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:38.639863 4732 raft_consensus.cc:2454] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b752fcd9b59a45d7be3e801be6365ecf in term 1.
I20260620 23:07:38.639863 4470 raft_consensus.cc:2454] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b752fcd9b59a45d7be3e801be6365ecf in term 1.
I20260620 23:07:38.640025 4534 leader_election.cc:302] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b752fcd9b59a45d7be3e801be6365ecf, fdc3a2c7ff534e9db7ce9785af743d9a; no voters:
I20260620 23:07:38.640110 4800 raft_consensus.cc:2781] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:38.640254 4800 raft_consensus.cc:686] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 1 LEADER]: Becoming Leader. State: Replica: b752fcd9b59a45d7be3e801be6365ecf, State: Running, Role: LEADER
I20260620 23:07:38.640399 4800 consensus_queue.cc:237] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } }
I20260620 23:07:38.641160 4328 catalog_manager.cc:5495] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf reported cstate change: term changed from 0 to 1, leader changed from <none> to b752fcd9b59a45d7be3e801be6365ecf (127.17.188.66). New cstate: current_term: 1 leader_uuid: "b752fcd9b59a45d7be3e801be6365ecf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b752fcd9b59a45d7be3e801be6365ecf" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 42343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 } health_report { overall_health: UNKNOWN } } }
W20260620 23:07:38.654891 4648 tablet.cc:2369] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:07:38.659159 4790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.656701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36260 (local address 127.17.188.66:42343)
0620 23:07:38.656768 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:38.656771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:38.656778 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:07:38.656806 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:38.656807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:38.656836 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:38.656915 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:07:38.657047 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.657573 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.658404 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:38.658605 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:38.658677 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:38.658845 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:07:38.658933 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:07:38.659037 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:38.659068 (+ 31us) server_negotiation.cc:299] Negotiation successful
0620 23:07:38.659075 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:07:38.660760 4470 raft_consensus.cc:1261] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694 [term 1 FOLLOWER]: Refusing update from remote peer b752fcd9b59a45d7be3e801be6365ecf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:38.660768 4732 raft_consensus.cc:1261] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [term 1 FOLLOWER]: Refusing update from remote peer b752fcd9b59a45d7be3e801be6365ecf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:38.661132 4802 consensus_queue.cc:1035] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e8908db17f4eb0b6ee49b49deb5694" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:38.661232 4800 consensus_queue.cc:1035] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdc3a2c7ff534e9db7ce9785af743d9a" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:38.663461 4805 mvcc.cc:205] Tried to move back new op lower bound from 7299059133071941632 to 7299059132991791104. Current Snapshot: MvccSnapshot[applied={T|T < 7299059133071941632}]
I20260620 23:07:38.663646 4806 mvcc.cc:205] Tried to move back new op lower bound from 7299059133071941632 to 7299059132991791104. Current Snapshot: MvccSnapshot[applied={T|T < 7299059133071941632}]
I20260620 23:07:38.663759 4807 mvcc.cc:205] Tried to move back new op lower bound from 7299059133071941632 to 7299059132991791104. Current Snapshot: MvccSnapshot[applied={T|T < 7299059133071941632}]
I20260620 23:07:38.665964 4327 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:38.666049 4327 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:38.666791 4327 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3abe11b3a774eddb275c4914a979aef: Sending DeleteTablet for 3 replicas of tablet 3685c79c9a404333ae0bce02523eab4b
I20260620 23:07:38.667116 4581 tablet_service.cc:1514] Processing DeleteTablet for tablet 3685c79c9a404333ae0bce02523eab4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:38 UTC) from {username='slave'} at 127.0.0.1:36248
I20260620 23:07:38.667114 4450 tablet_service.cc:1514] Processing DeleteTablet for tablet 3685c79c9a404333ae0bce02523eab4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:38 UTC) from {username='slave'} at 127.0.0.1:38354
I20260620 23:07:38.667341 4815 tablet_replica.cc:330] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: stopping tablet replica
I20260620 23:07:38.667384 4712 tablet_service.cc:1514] Processing DeleteTablet for tablet 3685c79c9a404333ae0bce02523eab4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:38 UTC) from {username='slave'} at 127.0.0.1:52730
I20260620 23:07:38.667433 4815 raft_consensus.cc:2229] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:07:38.667510 4815 raft_consensus.cc:2258] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:38.667550 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 4388
I20260620 23:07:38.667654 4816 tablet_replica.cc:330] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694: stopping tablet replica
I20260620 23:07:38.667717 4817 tablet_replica.cc:330] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: stopping tablet replica
I20260620 23:07:38.667811 4817 raft_consensus.cc:2229] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:38.667897 4816 raft_consensus.cc:2229] T 3685c79c9a404333ae0bce02523eab4b P 25e8908db17f4eb0b6ee49b49deb5694 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:38.668008 4817 raft_consensus.cc:2258] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:38.668249 4815 ts_tablet_manager.cc:1905] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:38.668732 4817 ts_tablet_manager.cc:1905] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:38.669786 4815 ts_tablet_manager.cc:1918] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:38.669860 4815 log.cc:1196] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/3685c79c9a404333ae0bce02523eab4b
I20260620 23:07:38.670143 4815 ts_tablet_manager.cc:1939] T 3685c79c9a404333ae0bce02523eab4b P b752fcd9b59a45d7be3e801be6365ecf: Deleting consensus metadata
I20260620 23:07:38.670517 4313 catalog_manager.cc:4841] TS b752fcd9b59a45d7be3e801be6365ecf (127.17.188.66:42343): tablet 3685c79c9a404333ae0bce02523eab4b (table test-table [id=9dd616c0e2fc443d953a0524ca7d7faf]) successfully deleted
I20260620 23:07:38.670914 4817 ts_tablet_manager.cc:1918] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:38.671000 4817 log.cc:1196] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/3685c79c9a404333ae0bce02523eab4b
I20260620 23:07:38.671263 4817 ts_tablet_manager.cc:1939] T 3685c79c9a404333ae0bce02523eab4b P fdc3a2c7ff534e9db7ce9785af743d9a: Deleting consensus metadata
I20260620 23:07:38.671629 4312 catalog_manager.cc:4841] TS fdc3a2c7ff534e9db7ce9785af743d9a (127.17.188.67:45279): tablet 3685c79c9a404333ae0bce02523eab4b (table test-table [id=9dd616c0e2fc443d953a0524ca7d7faf]) successfully deleted
W20260620 23:07:38.674852 4312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:33237 (error 108)
I20260620 23:07:38.675045 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 4519
I20260620 23:07:38.675287 4789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:38.675179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:33237 (local address 127.0.0.1:38360)
0620 23:07:38.675221 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:38.675253 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:33237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260620 23:07:38.675381 4312 catalog_manager.cc:4584] TS 25e8908db17f4eb0b6ee49b49deb5694 (127.17.188.65:33237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3685c79c9a404333ae0bce02523eab4b: Network error: Client connection negotiation failed: client connection to 127.17.188.65:33237: connect: Connection refused (error 111)
I20260620 23:07:38.681747 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 4650
I20260620 23:07:38.686100 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 4297
2026-06-20T23:07:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (697 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:38 dist-test-slave-zpss krb5kdc[4821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:38 dist-test-slave-zpss krb5kdc[4821](info): set up 2 sockets
Jun 20 23:07:38 dist-test-slave-zpss krb5kdc[4821](info): commencing operation
krb5kdc: starting...
W20260620 23:07:39.716952 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:39 dist-test-slave-zpss krb5kdc[4821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:07:39.824961 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:39115
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:38445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:39115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:39.903465 4834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:39.903750 4834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:39.903854 4834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:39.905081 4834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:39.905145 4834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:39.905172 4834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:39.905201 4834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:39.905225 4834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:39.906791 4834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:38445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:39115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:39115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:39.907058 4834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:39.907300 4834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:07:39.912111 4834 server_base.cc:1034] running on GCE node
W20260620 23:07:39.912057 4840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:39.912104 4842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:39.912057 4839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:39.912488 4834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:39.912802 4834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:39.913942 4834 hybrid_clock.cc:648] HybridClock initialized: now 1781996859913932 us; error 29 us; skew 500 ppm
Jun 20 23:07:39 dist-test-slave-zpss krb5kdc[4821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:39.916406 4834 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:07:39.916831 4834 webserver.cc:460] Webserver started at http://127.17.188.126:36147/ using document root <none> and password file <none>
I20260620 23:07:39.917088 4834 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:39.917133 4834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:39.917291 4834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:39.918421 4834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "2ec81dbd583d49cba3c817fe30bf333a"
format_stamp: "Formatted at 2026-06-20 23:07:39 on dist-test-slave-zpss"
server_key: "99679fb6017ce79fe1f2a175968038e6"
server_key_iv: "f4920294f9dc83901efe969185e82c4e"
server_key_version: "encryptionkey@0"
I20260620 23:07:39.918790 4834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "2ec81dbd583d49cba3c817fe30bf333a"
format_stamp: "Formatted at 2026-06-20 23:07:39 on dist-test-slave-zpss"
server_key: "99679fb6017ce79fe1f2a175968038e6"
server_key_iv: "f4920294f9dc83901efe969185e82c4e"
server_key_version: "encryptionkey@0"
I20260620 23:07:39.920055 4834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:39.920758 4849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:39.920940 4834 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:39.921037 4834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "2ec81dbd583d49cba3c817fe30bf333a"
format_stamp: "Formatted at 2026-06-20 23:07:39 on dist-test-slave-zpss"
server_key: "99679fb6017ce79fe1f2a175968038e6"
server_key_iv: "f4920294f9dc83901efe969185e82c4e"
server_key_version: "encryptionkey@0"
I20260620 23:07:39.921121 4834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:39.927955 4834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:39.930240 4834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:39.930389 4834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:39.934130 4834 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:39115
I20260620 23:07:39.934177 4901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:39115 every 8 connection(s)
I20260620 23:07:39.934509 4834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:39.935310 4902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:39.937492 4902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a: Bootstrap starting.
I20260620 23:07:39.938056 4902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:39.938333 4902 log.cc:824] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:39.938984 4902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a: No bootstrap required, opened a new log
I20260620 23:07:39.939154 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 4834
I20260620 23:07:39.939283 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:39.939476 18161 external_mini_cluster.cc:1428] Setting key b34db59c2b56cdb5cbd88b5fbcaa12cc
I20260620 23:07:39.941172 4902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec81dbd583d49cba3c817fe30bf333a" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 39115 } }
I20260620 23:07:39.941408 4902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:39.941462 4902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec81dbd583d49cba3c817fe30bf333a, State: Initialized, Role: FOLLOWER
I20260620 23:07:39.941594 4902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec81dbd583d49cba3c817fe30bf333a" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 39115 } }
I20260620 23:07:39.941686 4902 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:39.941708 4902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:39.941794 4902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:39.942898 4902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec81dbd583d49cba3c817fe30bf333a" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 39115 } }
I20260620 23:07:39.943001 4902 leader_election.cc:302] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ec81dbd583d49cba3c817fe30bf333a; no voters:
I20260620 23:07:39.943143 4902 leader_election.cc:288] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:39.943197 4907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:39.943357 4902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:39.943360 4907 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [term 1 LEADER]: Becoming Leader. State: Replica: 2ec81dbd583d49cba3c817fe30bf333a, State: Running, Role: LEADER
I20260620 23:07:39.943579 4907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec81dbd583d49cba3c817fe30bf333a" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 39115 } }
I20260620 23:07:39.943929 4908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ec81dbd583d49cba3c817fe30bf333a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec81dbd583d49cba3c817fe30bf333a" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 39115 } } }
I20260620 23:07:39.944010 4908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:39.944185 4909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ec81dbd583d49cba3c817fe30bf333a. Latest consensus state: current_term: 1 leader_uuid: "2ec81dbd583d49cba3c817fe30bf333a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec81dbd583d49cba3c817fe30bf333a" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 39115 } } }
I20260620 23:07:39.944252 4909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:39.944221 4915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:39.944787 4915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:39.946590 4915 catalog_manager.cc:1347] Generated new cluster ID: 71074795f3ea4269adc688ceb24ee9e5
I20260620 23:07:39.946640 4915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:07:39 dist-test-slave-zpss krb5kdc[4821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:39.951573 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:39.939939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45066 (local address 127.17.188.126:39115)
0620 23:07:39.940115 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:39.940125 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:39.940157 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:07:39.941138 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:39.941147 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:39.941210 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:39.941416 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:39.942185 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:39.943611 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:39.944694 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:39.945234 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:39.945600 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:39.948032 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:39.948036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:39.948036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:39.948047 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:39.949969 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:39.950395 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:39.950398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:39.950398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:39.950516 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:39.950736 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:39.950737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:39.950737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:39.951002 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:39.951151 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:39.951277 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 23:07:39.951396 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260620 23:07:39.966709 4915 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:39.967193 4915 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:39.979444 4915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a: Generated new TSK 0
I20260620 23:07:39.979607 4915 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.188.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
Entry for principal kudu/127.17.188.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab.
I20260620 23:07:40.003659 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:39115
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:38445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:40.083784 4928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:40.083976 4928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:40.083998 4928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:40.085115 4928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:40.085155 4928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:40.085181 4928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:07:40.086591 4928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:38445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.65.keytab
--principal=kudu/127.17.188.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:39115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:40.086819 4928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:40.087050 4928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:40.091897 4936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:40.091991 4933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:40.092154 4928 server_base.cc:1034] running on GCE node
W20260620 23:07:40.092137 4934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:40.092458 4928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:40.092753 4928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:40.093930 4928 hybrid_clock.cc:648] HybridClock initialized: now 1781996860093909 us; error 30 us; skew 500 ppm
Jun 20 23:07:40 dist-test-slave-zpss krb5kdc[4821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996860, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:40.096370 4928 init.cc:377] Logged in from keytab as kudu/127.17.188.65@KRBTEST.COM (short username kudu)
I20260620 23:07:40.096783 4928 webserver.cc:460] Webserver started at http://127.17.188.65:42635/ using document root <none> and password file <none>
I20260620 23:07:40.096997 4928 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:40.097064 4928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:40.097214 4928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:40.098245 4928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "0b18223233734afd8bc2f6753c26c644"
format_stamp: "Formatted at 2026-06-20 23:07:40 on dist-test-slave-zpss"
server_key: "390145b29c44bc7bf9027238f797d4c9"
server_key_iv: "337ebae90a30a144858aa732e3302eab"
server_key_version: "encryptionkey@0"
I20260620 23:07:40.098588 4928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "0b18223233734afd8bc2f6753c26c644"
format_stamp: "Formatted at 2026-06-20 23:07:40 on dist-test-slave-zpss"
server_key: "390145b29c44bc7bf9027238f797d4c9"
server_key_iv: "337ebae90a30a144858aa732e3302eab"
server_key_version: "encryptionkey@0"
I20260620 23:07:40.099860 4928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:07:40.100543 4943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:40.100713 4928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:40.100808 4928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "0b18223233734afd8bc2f6753c26c644"
format_stamp: "Formatted at 2026-06-20 23:07:40 on dist-test-slave-zpss"
server_key: "390145b29c44bc7bf9027238f797d4c9"
server_key_iv: "337ebae90a30a144858aa732e3302eab"
server_key_version: "encryptionkey@0"
I20260620 23:07:40.100888 4928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:40.110036 4928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:40.111739 4928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:40.111908 4928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:40.112114 4928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:40.112397 4928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:40.112457 4928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:40.112496 4928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:40.112524 4928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:40.118170 4928 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:33489
I20260620 23:07:40.118221 5056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:33489 every 8 connection(s)
I20260620 23:07:40.118487 4928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:07:40.118687 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 4928
I20260620 23:07:40.118804 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:07:40.119045 18161 external_mini_cluster.cc:1428] Setting key 132b6f98b66e9651d3285812ddbdfee3
Jun 20 23:07:40 dist-test-slave-zpss krb5kdc[4821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996860, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.65@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.188.66@KRBTEST.COM; defaulting to no policy
I20260620 23:07:40.128211 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.119422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:40187 (local address 127.17.188.126:39115)
0620 23:07:40.119465 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.119467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.119476 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:07:40.119936 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.119937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.119952 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.120007 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:40.120913 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.121503 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.122597 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.122962 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.123039 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.125516 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:40.125517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:40.125517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:40.125524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:40.126898 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:40.127378 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:40.127379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:40.127380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:40.127435 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:40.127692 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:40.127693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:40.127693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:40.127804 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:40.127857 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.128075 (+ 218us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.128142 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:07:40.128356 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.119147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:39115 (local address 127.17.188.65:40187)
0620 23:07:40.119313 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.119352 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:40.119835 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:40.120080 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:40.120083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:40.120623 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:40.120808 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.120812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.121592 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.121594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.122493 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.122496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.123046 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.123047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.123169 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.123660 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:40.123667 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:40.125419 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:40.126981 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:40.126988 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:40.126990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:40.127266 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:40.127504 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:40.127505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:40.127506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:40.127635 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:40.127866 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:40.127870 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:40.127973 (+ 103us) client_negotiation.cc:769] Sending connection context
0620 23:07:40.128051 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 23:07:40.128158 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260620 23:07:40.128877 5057 heartbeater.cc:347] Connected to a master server at 127.17.188.126:39115
I20260620 23:07:40.128963 5057 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:40.129087 5057 heartbeater.cc:511] Master 127.17.188.126:39115 requested a full tablet report, sending...
I20260620 23:07:40.129572 4866 ts_manager.cc:194] Registered new tserver with Master: 0b18223233734afd8bc2f6753c26c644 (127.17.188.65:33489)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.66@KRBTEST.COM" created.
I20260620 23:07:40.130187 4866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.65@KRBTEST.COM'} at 127.17.188.65:40187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
Entry for principal kudu/127.17.188.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab.
I20260620 23:07:40.141788 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:39115
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:38445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:40.218398 5062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:40.218596 5062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:40.218631 5062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:40.219689 5062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:40.219750 5062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:40.219790 5062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:07:40.221187 5062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:38445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.66.keytab
--principal=kudu/127.17.188.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:39115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:40.221464 5062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:40.221673 5062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:40.226498 5068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:40.226486 5067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:40.226464 5070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:40.226657 5062 server_base.cc:1034] running on GCE node
I20260620 23:07:40.226857 5062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:40.227116 5062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:40.228266 5062 hybrid_clock.cc:648] HybridClock initialized: now 1781996860228278 us; error 53 us; skew 500 ppm
Jun 20 23:07:40 dist-test-slave-zpss krb5kdc[4821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996860, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:40.230636 5062 init.cc:377] Logged in from keytab as kudu/127.17.188.66@KRBTEST.COM (short username kudu)
I20260620 23:07:40.231036 5062 webserver.cc:460] Webserver started at http://127.17.188.66:42425/ using document root <none> and password file <none>
I20260620 23:07:40.231256 5062 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:40.231323 5062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:40.231474 5062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:40.232395 5062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "d1169656fad2420395624c56c23bc515"
format_stamp: "Formatted at 2026-06-20 23:07:40 on dist-test-slave-zpss"
server_key: "8ef52a9fd61716bc4598a5b9d5758b05"
server_key_iv: "46d0450bbf105013f1be7b24bf5962f8"
server_key_version: "encryptionkey@0"
I20260620 23:07:40.232738 5062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "d1169656fad2420395624c56c23bc515"
format_stamp: "Formatted at 2026-06-20 23:07:40 on dist-test-slave-zpss"
server_key: "8ef52a9fd61716bc4598a5b9d5758b05"
server_key_iv: "46d0450bbf105013f1be7b24bf5962f8"
server_key_version: "encryptionkey@0"
I20260620 23:07:40.234014 5062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:07:40.234678 5077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:40.234833 5062 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:40.234925 5062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "d1169656fad2420395624c56c23bc515"
format_stamp: "Formatted at 2026-06-20 23:07:40 on dist-test-slave-zpss"
server_key: "8ef52a9fd61716bc4598a5b9d5758b05"
server_key_iv: "46d0450bbf105013f1be7b24bf5962f8"
server_key_version: "encryptionkey@0"
I20260620 23:07:40.235003 5062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:40.245970 5062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:40.247661 5062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:40.247809 5062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:40.248003 5062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:40.248270 5062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:40.248323 5062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:40.248359 5062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:40.248389 5062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:40.254093 5062 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:34215
I20260620 23:07:40.254123 5190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:34215 every 8 connection(s)
I20260620 23:07:40.254405 5062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:07:40.256008 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 5062
I20260620 23:07:40.256106 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:07:40.256294 18161 external_mini_cluster.cc:1428] Setting key a4df00b5fc3d3c966fb28f93ff5fa12f
Jun 20 23:07:40 dist-test-slave-zpss krb5kdc[4821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996860, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.66@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:40.263826 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.255089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:39715 (local address 127.17.188.126:39115)
0620 23:07:40.255174 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.255177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.255355 (+ 178us) server_negotiation.cc:407] Connection header received
0620 23:07:40.255842 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.255843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.255855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.255917 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:40.257028 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.257587 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.258566 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.258775 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.258851 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.261078 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:40.261080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:40.261080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:40.261087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:40.262458 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:40.262881 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:40.262882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:40.262883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:40.262942 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:40.263208 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:40.263209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:40.263209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:40.263316 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:40.263388 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.263657 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.263738 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
WARNING: no policy specified for kudu/127.17.188.67@KRBTEST.COM; defaulting to no policy
I20260620 23:07:40.264014 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.255044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:39115 (local address 127.17.188.66:39715)
0620 23:07:40.255220 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.255266 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:40.255727 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:40.255999 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:40.256007 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:40.256682 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:40.256938 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.256943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.257675 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.257676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.258470 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.258472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.258857 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.258858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.259016 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.259482 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:40.259489 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:40.260982 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:40.262552 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:40.262559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:40.262561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:40.262802 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:40.263011 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:40.263012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:40.263012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:40.263129 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:40.263391 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:40.263398 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:40.263527 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 23:07:40.263625 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 23:07:40.263783 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260620 23:07:40.264451 5191 heartbeater.cc:347] Connected to a master server at 127.17.188.126:39115
I20260620 23:07:40.264552 5191 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:40.264724 5191 heartbeater.cc:511] Master 127.17.188.126:39115 requested a full tablet report, sending...
I20260620 23:07:40.265115 4866 ts_manager.cc:194] Registered new tserver with Master: d1169656fad2420395624c56c23bc515 (127.17.188.66:34215)
I20260620 23:07:40.265532 4866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.66@KRBTEST.COM'} at 127.17.188.66:39715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
Entry for principal kudu/127.17.188.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab.
I20260620 23:07:40.278129 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:39115
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:38445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:40.354941 5196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:40.355110 5196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:40.355131 5196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:40.356280 5196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:40.356319 5196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:40.356367 5196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:07:40.357708 5196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:38445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.67.keytab
--principal=kudu/127.17.188.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:39115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:40.357930 5196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:40.358139 5196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:40.362798 5204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:40.362819 5202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:40.362901 5196 server_base.cc:1034] running on GCE node
W20260620 23:07:40.362870 5201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:40.363174 5196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:40.363448 5196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:40.364614 5196 hybrid_clock.cc:648] HybridClock initialized: now 1781996860364601 us; error 24 us; skew 500 ppm
Jun 20 23:07:40 dist-test-slave-zpss krb5kdc[4821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996860, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:40.366911 5196 init.cc:377] Logged in from keytab as kudu/127.17.188.67@KRBTEST.COM (short username kudu)
I20260620 23:07:40.367288 5196 webserver.cc:460] Webserver started at http://127.17.188.67:42877/ using document root <none> and password file <none>
I20260620 23:07:40.367486 5196 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:40.367549 5196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:40.367688 5196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:40.368652 5196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "5ad1318c63c442f2bf1c7412bb3fafec"
format_stamp: "Formatted at 2026-06-20 23:07:40 on dist-test-slave-zpss"
server_key: "1a02d965ee2060e7b6635462e945d786"
server_key_iv: "67d77e2c9183cf9e6b8694fabb2baf04"
server_key_version: "encryptionkey@0"
I20260620 23:07:40.369024 5196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "5ad1318c63c442f2bf1c7412bb3fafec"
format_stamp: "Formatted at 2026-06-20 23:07:40 on dist-test-slave-zpss"
server_key: "1a02d965ee2060e7b6635462e945d786"
server_key_iv: "67d77e2c9183cf9e6b8694fabb2baf04"
server_key_version: "encryptionkey@0"
I20260620 23:07:40.370312 5196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:07:40.370996 5211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:40.371155 5196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:40.371253 5196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "5ad1318c63c442f2bf1c7412bb3fafec"
format_stamp: "Formatted at 2026-06-20 23:07:40 on dist-test-slave-zpss"
server_key: "1a02d965ee2060e7b6635462e945d786"
server_key_iv: "67d77e2c9183cf9e6b8694fabb2baf04"
server_key_version: "encryptionkey@0"
I20260620 23:07:40.371331 5196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:40.390290 5196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:40.391945 5196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:40.392097 5196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:40.392297 5196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:40.392570 5196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:40.392620 5196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:40.392658 5196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:40.392684 5196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:40.397959 5196 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:45001
I20260620 23:07:40.397994 5324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:45001 every 8 connection(s)
I20260620 23:07:40.398272 5196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:07:40.401932 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 5196
I20260620 23:07:40.402056 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:07:40.402230 18161 external_mini_cluster.cc:1428] Setting key 3028f34fc40a4acd9c497e48c36ffdac
Jun 20 23:07:40 dist-test-slave-zpss krb5kdc[4821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996860, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.67@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:40.406812 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.398860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:39745 (local address 127.17.188.126:39115)
0620 23:07:40.398918 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.398920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.399098 (+ 178us) server_negotiation.cc:407] Connection header received
0620 23:07:40.399476 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.399477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.399487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.399534 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:40.400327 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.400803 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.401768 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.401938 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.401986 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.404275 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:40.404277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:40.404277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:40.404284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:40.405762 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:40.406089 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:40.406090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:40.406090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:40.406137 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:40.406329 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:40.406329 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:40.406329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:40.406418 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:40.406465 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.406693 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.406751 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 23:07:40.406939 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.398837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:39115 (local address 127.17.188.67:39745)
0620 23:07:40.398982 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.399009 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:40.399405 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:40.399577 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:40.399580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:40.400069 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:40.400258 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.400262 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.400875 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.400876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.401695 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.401697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.402020 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.402021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.402164 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.402638 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:07:40.402646 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:40.404169 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:40.405838 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:40.405842 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:40.405844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:40.406029 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:40.406188 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:07:40.406189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:07:40.406189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:07:40.406278 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:07:40.406468 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:40.406472 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:40.406571 (+ 99us) client_negotiation.cc:769] Sending connection context
0620 23:07:40.406665 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 23:07:40.406771 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260620 23:07:40.407336 5325 heartbeater.cc:347] Connected to a master server at 127.17.188.126:39115
I20260620 23:07:40.407407 5325 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:40.407582 5325 heartbeater.cc:511] Master 127.17.188.126:39115 requested a full tablet report, sending...
I20260620 23:07:40.407936 4866 ts_manager.cc:194] Registered new tserver with Master: 5ad1318c63c442f2bf1c7412bb3fafec (127.17.188.67:45001)
I20260620 23:07:40.408236 4866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.188.67@KRBTEST.COM'} at 127.17.188.67:39745
I20260620 23:07:40.413882 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:07:40.420296 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.415200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.17.188.126:39115)
0620 23:07:40.415255 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.415257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.415381 (+ 124us) server_negotiation.cc:407] Connection header received
0620 23:07:40.415405 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.415406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.415414 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.415460 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:40.415958 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.416405 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.417027 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.417179 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.417227 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.418226 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:40.418227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:40.418227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:40.418233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:40.419469 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:40.419711 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:40.419712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:40.419712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:40.419748 (+ 36us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:40.419878 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:40.419878 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:40.419879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:40.419984 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:40.420050 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.420133 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.420209 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:07:40.421782 4866 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:07:40.423472 4866 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:07:40.429260 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.425520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:34215 (local address 127.0.0.1:36858)
0620 23:07:40.425999 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.426007 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:40.426077 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:40.426254 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:40.426255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:40.426257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:40.426288 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.426289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.427093 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.427094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.428197 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.428200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.429009 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.429011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.429078 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.429089 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 23:07:40.429152 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 23:07:40.429198 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":45,"threads_started":1}
I20260620 23:07:40.429284 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.425712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36858 (local address 127.17.188.66:34215)
0620 23:07:40.425879 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.425882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.426134 (+ 252us) server_negotiation.cc:407] Connection header received
0620 23:07:40.426175 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.426178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.426193 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.426259 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:40.426331 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.426997 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.428303 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.428933 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.429009 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.429060 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.429168 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.429205 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260620 23:07:40.429557 5334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.425508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:33489 (local address 127.0.0.1:36298)
0620 23:07:40.425841 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.425850 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:40.425898 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:40.426073 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:40.426074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:40.426076 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:40.426130 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.426132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.427508 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.427509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.428355 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.428359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.429301 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.429302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.429381 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.429384 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:40.429428 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:07:40.429462 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":33,"spinlock_wait_cycles":768,"thread_start_us":73,"threads_started":1}
I20260620 23:07:40.429692 5336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.425649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43672 (local address 127.17.188.67:45001)
0620 23:07:40.425803 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.425807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.425816 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:07:40.425850 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.425852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.425868 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.425936 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:40.426111 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.427422 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.428669 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.429280 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.429361 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.429407 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.429596 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.429631 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
I20260620 23:07:40.429770 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.425637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.17.188.65:33489)
0620 23:07:40.425847 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.425852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.425892 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:07:40.425965 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.425967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.425981 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.426066 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:40.426240 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.427380 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.428448 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.429143 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.429218 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.429266 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.429659 (+ 393us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.429698 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
I20260620 23:07:40.429838 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.425507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:45001 (local address 127.0.0.1:43672)
0620 23:07:40.425654 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.425691 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:40.425749 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:40.425966 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:40.425969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:40.425971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:40.426032 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.426034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.427512 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.427513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.428572 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.428574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.429436 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.429437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.429480 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.429483 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:40.429524 (+ 41us) client_negotiation.cc:240] Negotiation successful
0620 23:07:40.429562 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":28,"thread_start_us":50,"threads_started":1}
I20260620 23:07:40.430182 5125 tablet_service.cc:1467] Processing CreateTablet for tablet 280b0d9d7d624cebb066bddda57d3543 (DEFAULT_TABLE table=test-table [id=f94ca7880d054268a93fe1bfafb971e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:40.430416 5259 tablet_service.cc:1467] Processing CreateTablet for tablet 280b0d9d7d624cebb066bddda57d3543 (DEFAULT_TABLE table=test-table [id=f94ca7880d054268a93fe1bfafb971e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:40.430513 5125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 280b0d9d7d624cebb066bddda57d3543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:40.430676 5259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 280b0d9d7d624cebb066bddda57d3543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:40.433655 5340 tablet_bootstrap.cc:492] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515: Bootstrap starting.
I20260620 23:07:40.433738 4991 tablet_service.cc:1467] Processing CreateTablet for tablet 280b0d9d7d624cebb066bddda57d3543 (DEFAULT_TABLE table=test-table [id=f94ca7880d054268a93fe1bfafb971e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:40.434054 4991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 280b0d9d7d624cebb066bddda57d3543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:40.434275 5340 tablet_bootstrap.cc:654] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:40.434891 5340 log.cc:824] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:40.435223 5339 tablet_bootstrap.cc:492] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: Bootstrap starting.
I20260620 23:07:40.435536 5340 tablet_bootstrap.cc:492] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515: No bootstrap required, opened a new log
I20260620 23:07:40.435580 5340 ts_tablet_manager.cc:1397] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:07:40.435822 5339 tablet_bootstrap.cc:654] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:40.436349 5339 log.cc:824] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:40.436977 5340 raft_consensus.cc:348] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } }
I20260620 23:07:40.437088 5340 raft_consensus.cc:374] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:40.437107 5340 raft_consensus.cc:729] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1169656fad2420395624c56c23bc515, State: Initialized, Role: FOLLOWER
I20260620 23:07:40.437103 5339 tablet_bootstrap.cc:492] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: No bootstrap required, opened a new log
I20260620 23:07:40.437212 5339 ts_tablet_manager.cc:1397] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:40.437215 5340 consensus_queue.cc:260] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } }
I20260620 23:07:40.437420 5343 tablet_bootstrap.cc:492] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644: Bootstrap starting.
I20260620 23:07:40.437503 5340 ts_tablet_manager.cc:1428] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 23:07:40.437615 5191 heartbeater.cc:503] Master 127.17.188.126:39115 was elected leader, sending a full tablet report...
I20260620 23:07:40.438006 5343 tablet_bootstrap.cc:654] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:40.438529 5343 log.cc:824] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:40.439167 5343 tablet_bootstrap.cc:492] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644: No bootstrap required, opened a new log
I20260620 23:07:40.439213 5343 ts_tablet_manager.cc:1397] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:40.439306 5339 raft_consensus.cc:348] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } }
I20260620 23:07:40.439455 5339 raft_consensus.cc:374] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:40.439498 5339 raft_consensus.cc:729] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ad1318c63c442f2bf1c7412bb3fafec, State: Initialized, Role: FOLLOWER
I20260620 23:07:40.439599 5339 consensus_queue.cc:260] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } }
I20260620 23:07:40.439850 5339 ts_tablet_manager.cc:1428] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:07:40.439869 5325 heartbeater.cc:503] Master 127.17.188.126:39115 was elected leader, sending a full tablet report...
I20260620 23:07:40.441107 5343 raft_consensus.cc:348] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } }
I20260620 23:07:40.441226 5343 raft_consensus.cc:374] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:40.441255 5343 raft_consensus.cc:729] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b18223233734afd8bc2f6753c26c644, State: Initialized, Role: FOLLOWER
I20260620 23:07:40.441385 5343 consensus_queue.cc:260] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } }
I20260620 23:07:40.441664 5343 ts_tablet_manager.cc:1428] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260620 23:07:40.441740 5057 heartbeater.cc:503] Master 127.17.188.126:39115 was elected leader, sending a full tablet report...
W20260620 23:07:40.504933 5192 tablet.cc:2369] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:07:40.619341 5058 tablet.cc:2369] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:07:40.648746 5326 tablet.cc:2369] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:07:40.819361 5346 raft_consensus.cc:482] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:40.819489 5346 raft_consensus.cc:504] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } }
I20260620 23:07:40.819800 5346 leader_election.cc:288] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b18223233734afd8bc2f6753c26c644 (127.17.188.65:33489), d1169656fad2420395624c56c23bc515 (127.17.188.66:34215)
I20260620 23:07:40.822810 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.820087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:38015 (local address 127.17.188.65:33489)
0620 23:07:40.820129 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.820131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.820174 (+ 43us) server_negotiation.cc:407] Connection header received
0620 23:07:40.820199 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.820200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.820208 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.820251 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:40.820440 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.821038 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.822064 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.822447 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.822500 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.822506 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.822675 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.822716 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:07:40.822811 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.820005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:33489 (local address 127.17.188.67:38015)
0620 23:07:40.820049 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.820062 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:40.820108 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:40.820285 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:40.820285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:40.820287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:40.820362 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.820364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.821117 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.821118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.821977 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.821979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.822519 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.822520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.822603 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.822606 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:40.822659 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 23:07:40.822707 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260620 23:07:40.822896 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.820057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:35875 (local address 127.17.188.66:34215)
0620 23:07:40.820112 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.820113 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.820419 (+ 306us) server_negotiation.cc:407] Connection header received
0620 23:07:40.820484 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.820485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.820499 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.820544 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:40.820705 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.821185 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.822161 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.822533 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.822604 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.822611 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.822790 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.822828 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 23:07:40.822932 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.820005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:34215 (local address 127.17.188.67:35875)
0620 23:07:40.820351 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.820360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:40.820431 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:40.820566 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:40.820567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:40.820568 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:40.820632 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.820634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.821273 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.821274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.822074 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:40.822076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.822612 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.822613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:40.822696 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.822700 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:07:40.822757 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:07:40.822803 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":149,"thread_start_us":115,"threads_started":1}
I20260620 23:07:40.823200 5145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "280b0d9d7d624cebb066bddda57d3543" candidate_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1169656fad2420395624c56c23bc515" is_pre_election: true
I20260620 23:07:40.823167 5011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "280b0d9d7d624cebb066bddda57d3543" candidate_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18223233734afd8bc2f6753c26c644" is_pre_election: true
I20260620 23:07:40.823335 5145 raft_consensus.cc:2454] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ad1318c63c442f2bf1c7412bb3fafec in term 0.
I20260620 23:07:40.823335 5011 raft_consensus.cc:2454] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ad1318c63c442f2bf1c7412bb3fafec in term 0.
I20260620 23:07:40.823556 5213 leader_election.cc:302] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ad1318c63c442f2bf1c7412bb3fafec, d1169656fad2420395624c56c23bc515; no voters:
I20260620 23:07:40.823674 5346 raft_consensus.cc:2781] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:07:40.823725 5346 raft_consensus.cc:482] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:07:40.823750 5346 raft_consensus.cc:3037] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:40.824544 5346 raft_consensus.cc:504] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } }
I20260620 23:07:40.824677 5346 leader_election.cc:288] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [CANDIDATE]: Term 1 election: Requested vote from peers 0b18223233734afd8bc2f6753c26c644 (127.17.188.65:33489), d1169656fad2420395624c56c23bc515 (127.17.188.66:34215)
I20260620 23:07:40.824832 5145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "280b0d9d7d624cebb066bddda57d3543" candidate_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1169656fad2420395624c56c23bc515"
I20260620 23:07:40.824832 5011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "280b0d9d7d624cebb066bddda57d3543" candidate_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18223233734afd8bc2f6753c26c644"
I20260620 23:07:40.824913 5145 raft_consensus.cc:3037] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:40.824913 5011 raft_consensus.cc:3037] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:40.825858 5145 raft_consensus.cc:2454] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ad1318c63c442f2bf1c7412bb3fafec in term 1.
I20260620 23:07:40.825858 5011 raft_consensus.cc:2454] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ad1318c63c442f2bf1c7412bb3fafec in term 1.
I20260620 23:07:40.826021 5213 leader_election.cc:302] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ad1318c63c442f2bf1c7412bb3fafec, d1169656fad2420395624c56c23bc515; no voters:
I20260620 23:07:40.826172 5346 raft_consensus.cc:2781] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:40.826318 5346 raft_consensus.cc:686] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 1 LEADER]: Becoming Leader. State: Replica: 5ad1318c63c442f2bf1c7412bb3fafec, State: Running, Role: LEADER
I20260620 23:07:40.826426 5346 consensus_queue.cc:237] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } }
I20260620 23:07:40.827248 4865 catalog_manager.cc:5495] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ad1318c63c442f2bf1c7412bb3fafec (127.17.188.67). New cstate: current_term: 1 leader_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ad1318c63c442f2bf1c7412bb3fafec" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 45001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 } health_report { overall_health: UNKNOWN } } }
I20260620 23:07:40.872303 5336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.870121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.17.188.67:45001)
0620 23:07:40.870180 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:40.870182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:40.870211 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:07:40.870257 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:40.870257 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:40.870272 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:40.870322 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:07:40.870424 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.870864 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.871615 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:40.871805 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:40.871853 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:40.871977 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:07:40.872070 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:07:40.872129 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:40.872161 (+ 32us) server_negotiation.cc:299] Negotiation successful
0620 23:07:40.872176 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:07:40.874050 5011 raft_consensus.cc:1261] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [term 1 FOLLOWER]: Refusing update from remote peer 5ad1318c63c442f2bf1c7412bb3fafec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:40.874050 5145 raft_consensus.cc:1261] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [term 1 FOLLOWER]: Refusing update from remote peer 5ad1318c63c442f2bf1c7412bb3fafec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:40.874418 5349 consensus_queue.cc:1035] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1169656fad2420395624c56c23bc515" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 34215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:40.874596 5346 consensus_queue.cc:1035] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b18223233734afd8bc2f6753c26c644" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 33489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:40.876593 5352 mvcc.cc:205] Tried to move back new op lower bound from 7299059142137364480 to 7299059141945536512. Current Snapshot: MvccSnapshot[applied={T|T < 7299059142137364480}]
I20260620 23:07:40.877346 5353 mvcc.cc:205] Tried to move back new op lower bound from 7299059142137364480 to 7299059141945536512. Current Snapshot: MvccSnapshot[applied={T|T < 7299059142137364480}]
I20260620 23:07:40.879348 4866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:40.879438 4866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:40.880149 4866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ec81dbd583d49cba3c817fe30bf333a: Sending DeleteTablet for 3 replicas of tablet 280b0d9d7d624cebb066bddda57d3543
I20260620 23:07:40.880492 5259 tablet_service.cc:1514] Processing DeleteTablet for tablet 280b0d9d7d624cebb066bddda57d3543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:40 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:43672
I20260620 23:07:40.880529 4991 tablet_service.cc:1514] Processing DeleteTablet for tablet 280b0d9d7d624cebb066bddda57d3543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:40 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260620 23:07:40.880796 5362 tablet_replica.cc:330] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644: stopping tablet replica
I20260620 23:07:40.880780 5125 tablet_service.cc:1514] Processing DeleteTablet for tablet 280b0d9d7d624cebb066bddda57d3543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:40 UTC) from {username='kudu', principal='kudu/127.17.188.126@KRBTEST.COM'} at 127.0.0.1:36858
I20260620 23:07:40.880889 5362 raft_consensus.cc:2229] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:40.880995 5362 raft_consensus.cc:2258] T 280b0d9d7d624cebb066bddda57d3543 P 0b18223233734afd8bc2f6753c26c644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:40.880986 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 4928
I20260620 23:07:40.881142 5361 tablet_replica.cc:330] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: stopping tablet replica
I20260620 23:07:40.881233 5361 raft_consensus.cc:2229] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:07:40.881651 5361 raft_consensus.cc:2258] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:40.882688 5361 ts_tablet_manager.cc:1905] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:40.883733 5363 tablet_replica.cc:330] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515: stopping tablet replica
I20260620 23:07:40.883868 5363 raft_consensus.cc:2229] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:40.884022 5363 raft_consensus.cc:2258] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:40.884639 5361 ts_tablet_manager.cc:1918] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:40.884716 5361 log.cc:1196] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/280b0d9d7d624cebb066bddda57d3543
I20260620 23:07:40.884968 5361 ts_tablet_manager.cc:1939] T 280b0d9d7d624cebb066bddda57d3543 P 5ad1318c63c442f2bf1c7412bb3fafec: Deleting consensus metadata
I20260620 23:07:40.885349 4850 catalog_manager.cc:4841] TS 5ad1318c63c442f2bf1c7412bb3fafec (127.17.188.67:45001): tablet 280b0d9d7d624cebb066bddda57d3543 (table test-table [id=f94ca7880d054268a93fe1bfafb971e6]) successfully deleted
W20260620 23:07:40.886252 4853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:33489 (error 108)
I20260620 23:07:40.886585 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 5062
I20260620 23:07:40.886629 5363 ts_tablet_manager.cc:1905] T 280b0d9d7d624cebb066bddda57d3543 P d1169656fad2420395624c56c23bc515: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:40.887400 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.887289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:33489 (local address 127.0.0.1:36304)
0620 23:07:40.887336 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.887370 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:33489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:07:40.887565 4853 catalog_manager.cc:4584] TS 0b18223233734afd8bc2f6753c26c644 (127.17.188.65:33489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 280b0d9d7d624cebb066bddda57d3543: Network error: Client connection negotiation failed: client connection to 127.17.188.65:33489: connect: Connection refused (error 111)
I20260620 23:07:40.891573 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:40.891466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:34215 (local address 127.0.0.1:36864)
0620 23:07:40.891517 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:40.891548 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.66:34215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 23:07:40.891999 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 5196
I20260620 23:07:40.896905 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 4834
2026-06-20T23:07:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2211 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T23:07:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:40Z Disabled control of system clock
I20260620 23:07:40.924230 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:38403
--webserver_interface=127.17.188.126
--webserver_port=0
--builtin_ntp_servers=127.17.188.84:37769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:38403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:07:41.001916 5368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:41.002097 5368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:41.002116 5368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:41.003242 5368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:41.003280 5368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:41.003293 5368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:41.003305 5368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:41.003356 5368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:41.004603 5368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:37769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:38403
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:38403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:41.004854 5368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:41.005123 5368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:41.009776 5376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:41.009927 5368 server_base.cc:1034] running on GCE node
W20260620 23:07:41.009778 5374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:41.009804 5373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:41.010251 5368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:41.010540 5368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:41.011698 5368 hybrid_clock.cc:648] HybridClock initialized: now 1781996861011684 us; error 45 us; skew 500 ppm
I20260620 23:07:41.012784 5368 webserver.cc:460] Webserver started at http://127.17.188.126:34875/ using document root <none> and password file <none>
I20260620 23:07:41.013005 5368 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:41.013070 5368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:41.013227 5368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:41.014247 5368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "cf20f2c0a0834f0b82728c2a5159e320"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "6a48a867c16367a02fa957ea02a26b43"
server_key_iv: "21cb91020a09fec176072086b5521f36"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.014595 5368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "cf20f2c0a0834f0b82728c2a5159e320"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "6a48a867c16367a02fa957ea02a26b43"
server_key_iv: "21cb91020a09fec176072086b5521f36"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.015774 5368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:07:41.016386 5382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.016528 5368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:41.016613 5368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "cf20f2c0a0834f0b82728c2a5159e320"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "6a48a867c16367a02fa957ea02a26b43"
server_key_iv: "21cb91020a09fec176072086b5521f36"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.016688 5368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:41.036152 5368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:41.036437 5368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:41.036598 5368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:41.040287 5368 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:38403
I20260620 23:07:41.040320 5434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:38403 every 8 connection(s)
I20260620 23:07:41.040634 5368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:41.041584 5435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:41.044049 5435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320: Bootstrap starting.
I20260620 23:07:41.044562 5435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:41.044819 5435 log.cc:824] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:41.045499 5435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320: No bootstrap required, opened a new log
I20260620 23:07:41.047086 5435 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf20f2c0a0834f0b82728c2a5159e320" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38403 } }
I20260620 23:07:41.047250 5435 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:41.047271 5435 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf20f2c0a0834f0b82728c2a5159e320, State: Initialized, Role: FOLLOWER
I20260620 23:07:41.047405 5435 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf20f2c0a0834f0b82728c2a5159e320" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38403 } }
I20260620 23:07:41.047483 5435 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:41.047508 5435 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:41.047544 5435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:41.047997 5435 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf20f2c0a0834f0b82728c2a5159e320" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38403 } }
I20260620 23:07:41.048087 5435 leader_election.cc:302] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf20f2c0a0834f0b82728c2a5159e320; no voters:
I20260620 23:07:41.048210 5435 leader_election.cc:288] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:41.048310 5438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:41.048451 5438 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [term 1 LEADER]: Becoming Leader. State: Replica: cf20f2c0a0834f0b82728c2a5159e320, State: Running, Role: LEADER
I20260620 23:07:41.048487 5435 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:41.048529 5438 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf20f2c0a0834f0b82728c2a5159e320" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38403 } }
I20260620 23:07:41.048568 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 5368
I20260620 23:07:41.048664 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:41.048838 5438 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf20f2c0a0834f0b82728c2a5159e320. Latest consensus state: current_term: 1 leader_uuid: "cf20f2c0a0834f0b82728c2a5159e320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf20f2c0a0834f0b82728c2a5159e320" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38403 } } }
I20260620 23:07:41.048895 18161 external_mini_cluster.cc:1428] Setting key 4062824deb494d8a05837dc028884169
I20260620 23:07:41.048914 5438 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:41.048925 5439 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf20f2c0a0834f0b82728c2a5159e320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf20f2c0a0834f0b82728c2a5159e320" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38403 } } }
I20260620 23:07:41.048979 5439 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:41.049873 5443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:41.050401 5443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:41.051944 5443 catalog_manager.cc:1347] Generated new cluster ID: 9cdfe51005914ebf977b46249bb4ae6d
I20260620 23:07:41.051994 5443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:41.054903 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.049534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.17.188.126:38403)
0620 23:07:41.049754 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.049765 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.049804 (+ 39us) server_negotiation.cc:407] Connection header received
0620 23:07:41.049925 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.049932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.050007 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.050172 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:41.050444 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.051643 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.052708 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.053262 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.053671 (+ 409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.054093 (+ 422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:41.054097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:41.054097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:41.054282 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:41.054317 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:41.054329 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:41.054585 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.054613 (+ 28us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.054681 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260620 23:07:41.061102 5443 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:41.061640 5443 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:41.069981 5443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320: Generated new TSK 0
I20260620 23:07:41.070133 5443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:41.076061 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.65:0
--local_ip_for_outbound_sockets=127.17.188.65
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38403
--builtin_ntp_servers=127.17.188.84:37769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:07:41.152837 5459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:41.153041 5459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:41.153076 5459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:41.154259 5459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:41.154317 5459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:41.154358 5459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.65
I20260620 23:07:41.155619 5459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:37769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.188.65
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.65
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:41.155859 5459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:41.156141 5459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:41.160969 5465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:41.160986 5467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:41.160993 5464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:41.161115 5459 server_base.cc:1034] running on GCE node
I20260620 23:07:41.161412 5459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:41.161679 5459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:41.162859 5459 hybrid_clock.cc:648] HybridClock initialized: now 1781996861162846 us; error 27 us; skew 500 ppm
I20260620 23:07:41.164222 5459 webserver.cc:460] Webserver started at http://127.17.188.65:42219/ using document root <none> and password file <none>
I20260620 23:07:41.164460 5459 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:41.164528 5459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:41.164664 5459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:41.166064 5459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/data/instance:
uuid: "f34cf33f244e41daaea566ceca8f6bb0"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "fb5045081755090640e5174ce9f78faf"
server_key_iv: "3dec65adff594acaf13a08263428ea67"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.166461 5459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance:
uuid: "f34cf33f244e41daaea566ceca8f6bb0"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "fb5045081755090640e5174ce9f78faf"
server_key_iv: "3dec65adff594acaf13a08263428ea67"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.168071 5459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:07:41.168952 5473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.169137 5459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:41.169231 5459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/wal
uuid: "f34cf33f244e41daaea566ceca8f6bb0"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "fb5045081755090640e5174ce9f78faf"
server_key_iv: "3dec65adff594acaf13a08263428ea67"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.169304 5459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:41.185529 5459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:41.185782 5459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:41.185886 5459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:41.186074 5459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:41.186340 5459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:41.186371 5459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.186394 5459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:41.186414 5459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.192610 5459 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.65:43323
I20260620 23:07:41.192662 5586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.65:43323 every 8 connection(s)
I20260620 23:07:41.193040 5459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/data/info.pb
I20260620 23:07:41.198191 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.193809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:37397 (local address 127.17.188.126:38403)
0620 23:07:41.193891 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.193893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.194130 (+ 237us) server_negotiation.cc:407] Connection header received
0620 23:07:41.194707 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.194708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.194721 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.194782 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:41.195393 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.195923 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.196872 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.197064 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.197129 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.197975 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:41.197976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:41.197976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:41.197980 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:41.197990 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:41.197993 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:41.198032 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.198107 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.198130 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 23:07:41.198381 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.193775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:38403 (local address 127.17.188.65:37397)
0620 23:07:41.193971 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:41.194001 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:41.194607 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:41.194834 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:41.194837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:41.194845 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:41.195306 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.195310 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.196005 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.196006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.196759 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.196761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.197148 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.197149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.197250 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.197848 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:07:41.197848 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:41.197876 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:07:41.197897 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:41.198032 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:41.198034 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:41.198036 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:07:41.198082 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 23:07:41.198166 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":116,"threads_started":1}
I20260620 23:07:41.198902 5587 heartbeater.cc:347] Connected to a master server at 127.17.188.126:38403
I20260620 23:07:41.198973 5587 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:41.199146 5587 heartbeater.cc:511] Master 127.17.188.126:38403 requested a full tablet report, sending...
I20260620 23:07:41.199611 5399 ts_manager.cc:194] Registered new tserver with Master: f34cf33f244e41daaea566ceca8f6bb0 (127.17.188.65:43323)
I20260620 23:07:41.200191 5399 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.65:37397
I20260620 23:07:41.200775 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 5459
I20260620 23:07:41.200870 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-0/wal/instance
I20260620 23:07:41.201076 18161 external_mini_cluster.cc:1428] Setting key d17a6f223d7f232c6acf3d66c3dda585
I20260620 23:07:41.201933 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.66:0
--local_ip_for_outbound_sockets=127.17.188.66
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38403
--builtin_ntp_servers=127.17.188.84:37769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:07:41.280473 5590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:41.280686 5590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:41.280725 5590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:41.281904 5590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:41.281967 5590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:41.282009 5590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.66
I20260620 23:07:41.283289 5590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:37769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.188.66
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.66
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:41.283527 5590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:41.283818 5590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:41.288973 5596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:41.289006 5598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:41.289003 5595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:41.289036 5590 server_base.cc:1034] running on GCE node
I20260620 23:07:41.289376 5590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:41.289626 5590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:41.290781 5590 hybrid_clock.cc:648] HybridClock initialized: now 1781996861290774 us; error 34 us; skew 500 ppm
I20260620 23:07:41.292034 5590 webserver.cc:460] Webserver started at http://127.17.188.66:42569/ using document root <none> and password file <none>
I20260620 23:07:41.292273 5590 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:41.292341 5590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:41.292466 5590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:41.293521 5590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/data/instance:
uuid: "bd007fa7889f41c8b8ad629d5b455285"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "3067d11b5204c0297f0fbfcf7208b3f4"
server_key_iv: "998a810e0df01d126e59af084118ff7c"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.293848 5590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance:
uuid: "bd007fa7889f41c8b8ad629d5b455285"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "3067d11b5204c0297f0fbfcf7208b3f4"
server_key_iv: "998a810e0df01d126e59af084118ff7c"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.295068 5590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:07:41.295709 5604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.295887 5590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:41.295979 5590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/wal
uuid: "bd007fa7889f41c8b8ad629d5b455285"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "3067d11b5204c0297f0fbfcf7208b3f4"
server_key_iv: "998a810e0df01d126e59af084118ff7c"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.296053 5590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:41.317485 5590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:41.317735 5590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:41.317868 5590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:41.318068 5590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:41.318359 5590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:41.318408 5590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.318447 5590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:41.318473 5590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.324026 5590 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.66:32975
I20260620 23:07:41.324117 5717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.66:32975 every 8 connection(s)
I20260620 23:07:41.324445 5590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/data/info.pb
I20260620 23:07:41.326412 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 5590
I20260620 23:07:41.326521 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/wal/instance
I20260620 23:07:41.326717 18161 external_mini_cluster.cc:1428] Setting key 1a4dfb31782eea03552595e5582299de
I20260620 23:07:41.327665 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.188.67:0
--local_ip_for_outbound_sockets=127.17.188.67
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38403
--builtin_ntp_servers=127.17.188.84:37769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:07:41.330396 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.325103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.66:53577 (local address 127.17.188.126:38403)
0620 23:07:41.325147 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.325149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.325444 (+ 295us) server_negotiation.cc:407] Connection header received
0620 23:07:41.325963 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.325964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.325978 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.326029 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:41.326740 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.327348 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.328830 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.329089 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.329163 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.330078 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:41.330079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:41.330079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:41.330084 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:41.330099 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:41.330103 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:41.330163 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.330239 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.330283 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:07:41.330488 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.325105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:38403 (local address 127.17.188.66:53577)
0620 23:07:41.325273 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:41.325349 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:41.325846 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:41.326092 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:41.326096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:41.326106 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:41.326647 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.326652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.327450 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.327451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.328747 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.328751 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.329195 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.329196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.329301 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.329952 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:07:41.329952 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:41.329978 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:07:41.329999 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:41.330164 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:41.330166 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:41.330168 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:07:41.330220 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 23:07:41.330313 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260620 23:07:41.330926 5718 heartbeater.cc:347] Connected to a master server at 127.17.188.126:38403
I20260620 23:07:41.331027 5718 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:41.331221 5718 heartbeater.cc:511] Master 127.17.188.126:38403 requested a full tablet report, sending...
I20260620 23:07:41.331579 5399 ts_manager.cc:194] Registered new tserver with Master: bd007fa7889f41c8b8ad629d5b455285 (127.17.188.66:32975)
I20260620 23:07:41.331954 5399 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.66:53577
W20260620 23:07:41.406143 5721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:41.406347 5721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:41.406379 5721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:41.407500 5721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:41.407559 5721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:41.407600 5721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.188.67
I20260620 23:07:41.408890 5721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:37769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.188.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.188.67
--webserver_port=0
--tserver_master_addrs=127.17.188.126:38403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.188.67
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:41.409117 5721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:41.409438 5721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:41.414155 5726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:41.414143 5727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:41.414145 5729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:41.414368 5721 server_base.cc:1034] running on GCE node
I20260620 23:07:41.414569 5721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:41.414759 5721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:41.415895 5721 hybrid_clock.cc:648] HybridClock initialized: now 1781996861415883 us; error 28 us; skew 500 ppm
I20260620 23:07:41.417207 5721 webserver.cc:460] Webserver started at http://127.17.188.67:35183/ using document root <none> and password file <none>
I20260620 23:07:41.417469 5721 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:41.417517 5721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:41.417658 5721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:41.418744 5721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/data/instance:
uuid: "088b347fe1544556a9c52207f86a4313"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "3ca7659e9a576d7190ce20d2065fc60d"
server_key_iv: "cc408bfc5dee475b22e75a4cffd9b59b"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.419095 5721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance:
uuid: "088b347fe1544556a9c52207f86a4313"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "3ca7659e9a576d7190ce20d2065fc60d"
server_key_iv: "cc408bfc5dee475b22e75a4cffd9b59b"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.420385 5721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:07:41.421041 5735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.421214 5721 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:41.421278 5721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/wal
uuid: "088b347fe1544556a9c52207f86a4313"
format_stamp: "Formatted at 2026-06-20 23:07:41 on dist-test-slave-zpss"
server_key: "3ca7659e9a576d7190ce20d2065fc60d"
server_key_iv: "cc408bfc5dee475b22e75a4cffd9b59b"
server_key_version: "encryptionkey@0"
I20260620 23:07:41.421386 5721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:41.438494 5721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:41.438772 5721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:41.438920 5721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:41.439132 5721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:07:41.439452 5721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:07:41.439507 5721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.439540 5721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:07:41.439566 5721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:41.445168 5721 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.67:40555
I20260620 23:07:41.445242 5848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.67:40555 every 8 connection(s)
I20260620 23:07:41.445562 5721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/data/info.pb
I20260620 23:07:41.450419 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.446285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.67:56955 (local address 127.17.188.126:38403)
0620 23:07:41.446326 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.446328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.446543 (+ 215us) server_negotiation.cc:407] Connection header received
0620 23:07:41.446972 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.446973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.446983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.447034 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:41.447609 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.448091 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.449093 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.449376 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.449434 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.450230 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:41.450231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:41.450231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:41.450236 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:41.450247 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:41.450251 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:41.450294 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.450341 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.450364 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:07:41.450639 5851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.446280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:38403 (local address 127.17.188.67:56955)
0620 23:07:41.446430 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:41.446461 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:41.446893 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:41.447082 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:41.447085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:41.447092 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:07:41.447544 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.447548 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.448202 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.448203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.449024 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.449026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.449453 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.449454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.449591 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.450107 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:07:41.450107 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:07:41.450133 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:07:41.450153 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:07:41.450294 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:07:41.450296 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:07:41.450298 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:07:41.450343 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 23:07:41.450410 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
I20260620 23:07:41.451205 5849 heartbeater.cc:347] Connected to a master server at 127.17.188.126:38403
I20260620 23:07:41.451305 5849 heartbeater.cc:464] Registering TS with master...
I20260620 23:07:41.451475 5849 heartbeater.cc:511] Master 127.17.188.126:38403 requested a full tablet report, sending...
I20260620 23:07:41.451807 5399 ts_manager.cc:194] Registered new tserver with Master: 088b347fe1544556a9c52207f86a4313 (127.17.188.67:40555)
I20260620 23:07:41.452143 5399 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.188.67:56955
I20260620 23:07:41.453146 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 5721
I20260620 23:07:41.453258 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/wal/instance
I20260620 23:07:41.453485 18161 external_mini_cluster.cc:1428] Setting key 168d4fb4b07d475bbae40af82c75ec27
I20260620 23:07:41.454336 18161 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:07:41.459327 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.455646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.17.188.126:38403)
0620 23:07:41.455680 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.455682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.455801 (+ 119us) server_negotiation.cc:407] Connection header received
0620 23:07:41.455874 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.455875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.455886 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.455941 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:41.456107 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.456867 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.458162 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.458367 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.458442 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.459103 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:41.459105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:41.459105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:41.459109 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:41.459121 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:41.459125 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:41.459171 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.459227 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.459247 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:07:41.461664 5399 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:07:41.463418 5399 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:07:41.469237 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.465221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:40555 (local address 127.0.0.1:33310)
0620 23:07:41.465398 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:41.465434 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:41.465493 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:41.465700 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:41.465703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:41.465706 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:41.465763 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.465765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.466894 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.466896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.468308 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.468310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.468956 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.468957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.469034 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.469047 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 23:07:41.469116 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:07:41.469166 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":68,"threads_started":1}
I20260620 23:07:41.469249 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.465400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.17.188.67:40555)
0620 23:07:41.465547 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.465551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.465563 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:07:41.465597 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.465599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.465614 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.465692 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:41.465829 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.466816 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.468394 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.468858 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.468942 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.469027 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.469129 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.469165 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260620 23:07:41.469565 5857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.465227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:32975 (local address 127.0.0.1:48784)
0620 23:07:41.465531 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:41.465541 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:41.465596 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:41.465996 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:41.465997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:41.465998 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:41.466040 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.466042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.467302 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.467302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.468328 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.468330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.469251 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.469252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.469345 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.469347 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:07:41.469396 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:07:41.469448 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":12,"thread_start_us":103,"threads_started":1}
I20260620 23:07:41.469736 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.465446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:43323 (local address 127.0.0.1:49442)
0620 23:07:41.466069 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:41.466079 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:41.466126 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:41.466330 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:41.466331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:41.466333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:41.466369 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.466370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.467526 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.467528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.468247 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.468248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.469248 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.469249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.469341 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.469346 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:07:41.469396 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 23:07:41.469446 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":6,"thread_start_us":72,"threads_started":1}
I20260620 23:07:41.469873 5859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.465358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.17.188.66:32975)
0620 23:07:41.465829 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.465834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.465844 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:07:41.465878 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.465880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.465893 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.465982 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:41.466123 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.467203 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.468391 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.469059 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.469129 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.469209 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.469675 (+ 466us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.469718 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":82,"threads_started":1}
I20260620 23:07:41.469889 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.465639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49442 (local address 127.17.188.65:43323)
0620 23:07:41.465829 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.465834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.466183 (+ 349us) server_negotiation.cc:407] Connection header received
0620 23:07:41.466225 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.466227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.466242 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.466317 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:41.466430 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.467441 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.468343 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.469060 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.469129 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.469210 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.469490 (+ 280us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.469740 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260620 23:07:41.470017 5783 tablet_service.cc:1467] Processing CreateTablet for tablet b861576712da4c619f23a2918337f00e (DEFAULT_TABLE table=test-table [id=09fca56bdbad4d0392a58c6e9ddb6183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:41.470379 5783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b861576712da4c619f23a2918337f00e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:41.470531 5521 tablet_service.cc:1467] Processing CreateTablet for tablet b861576712da4c619f23a2918337f00e (DEFAULT_TABLE table=test-table [id=09fca56bdbad4d0392a58c6e9ddb6183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:41.470528 5652 tablet_service.cc:1467] Processing CreateTablet for tablet b861576712da4c619f23a2918337f00e (DEFAULT_TABLE table=test-table [id=09fca56bdbad4d0392a58c6e9ddb6183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:07:41.470826 5652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b861576712da4c619f23a2918337f00e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:41.470827 5521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b861576712da4c619f23a2918337f00e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:41.472764 5863 tablet_bootstrap.cc:492] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: Bootstrap starting.
I20260620 23:07:41.473639 5863 tablet_bootstrap.cc:654] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:41.473965 5864 tablet_bootstrap.cc:492] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0: Bootstrap starting.
I20260620 23:07:41.474119 5863 log.cc:824] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:41.474632 5864 tablet_bootstrap.cc:654] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:41.474859 5863 tablet_bootstrap.cc:492] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: No bootstrap required, opened a new log
I20260620 23:07:41.474933 5863 ts_tablet_manager.cc:1397] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:41.474957 5864 log.cc:824] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:41.475270 5865 tablet_bootstrap.cc:492] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: Bootstrap starting.
I20260620 23:07:41.475648 5864 tablet_bootstrap.cc:492] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0: No bootstrap required, opened a new log
I20260620 23:07:41.475728 5864 ts_tablet_manager.cc:1397] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:41.476004 5865 tablet_bootstrap.cc:654] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:41.476300 5865 log.cc:824] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:41.477094 5865 tablet_bootstrap.cc:492] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: No bootstrap required, opened a new log
I20260620 23:07:41.477166 5865 ts_tablet_manager.cc:1397] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:07:41.477888 5864 raft_consensus.cc:348] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } }
I20260620 23:07:41.477845 5863 raft_consensus.cc:348] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } }
I20260620 23:07:41.478024 5863 raft_consensus.cc:374] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:41.478025 5864 raft_consensus.cc:374] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:41.478072 5864 raft_consensus.cc:729] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34cf33f244e41daaea566ceca8f6bb0, State: Initialized, Role: FOLLOWER
I20260620 23:07:41.478072 5863 raft_consensus.cc:729] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 088b347fe1544556a9c52207f86a4313, State: Initialized, Role: FOLLOWER
I20260620 23:07:41.478179 5864 consensus_queue.cc:260] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } }
I20260620 23:07:41.478235 5863 consensus_queue.cc:260] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } }
I20260620 23:07:41.478447 5864 ts_tablet_manager.cc:1428] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:07:41.478466 5863 ts_tablet_manager.cc:1428] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260620 23:07:41.478483 5587 heartbeater.cc:503] Master 127.17.188.126:38403 was elected leader, sending a full tablet report...
I20260620 23:07:41.478572 5849 heartbeater.cc:503] Master 127.17.188.126:38403 was elected leader, sending a full tablet report...
I20260620 23:07:41.479925 5865 raft_consensus.cc:348] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } }
I20260620 23:07:41.480033 5865 raft_consensus.cc:374] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:41.480052 5865 raft_consensus.cc:729] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd007fa7889f41c8b8ad629d5b455285, State: Initialized, Role: FOLLOWER
I20260620 23:07:41.480146 5865 consensus_queue.cc:260] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } }
I20260620 23:07:41.480372 5865 ts_tablet_manager.cc:1428] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 23:07:41.480453 5718 heartbeater.cc:503] Master 127.17.188.126:38403 was elected leader, sending a full tablet report...
W20260620 23:07:41.574946 5719 tablet.cc:2369] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:07:41.583190 5869 raft_consensus.cc:482] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:07:41.583293 5869 raft_consensus.cc:504] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } }
I20260620 23:07:41.583659 5869 leader_election.cc:288] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 088b347fe1544556a9c52207f86a4313 (127.17.188.67:40555), bd007fa7889f41c8b8ad629d5b455285 (127.17.188.66:32975)
I20260620 23:07:41.586474 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.583937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:53495 (local address 127.17.188.67:40555)
0620 23:07:41.583992 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.583994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.584004 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:07:41.584033 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.584039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.584056 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.584131 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:41.584294 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.584788 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.585791 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.586133 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.586220 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.586227 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.586367 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.586403 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:07:41.586505 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.583835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.67:40555 (local address 127.17.188.65:53495)
0620 23:07:41.583890 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:41.583902 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:41.583953 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:41.584137 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:41.584138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:41.584139 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:41.584217 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.584220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.584899 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.584900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.585705 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.585707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.586228 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.586229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.586298 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.586301 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:41.586362 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:07:41.586403 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":16}
I20260620 23:07:41.586568 5872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.583836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.66:32975 (local address 127.17.188.65:36275)
0620 23:07:41.584049 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:41.584056 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0620 23:07:41.584114 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:07:41.584258 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:07:41.584259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:07:41.584260 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:07:41.584330 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.584332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.584982 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.584983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.585762 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:07:41.585764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.586271 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.586272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:07:41.586344 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.586347 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:07:41.586410 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 23:07:41.586444 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":72,"thread_start_us":85,"threads_started":1}
I20260620 23:07:41.586508 5859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.583944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.188.65:36275 (local address 127.17.188.66:32975)
0620 23:07:41.583987 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.583989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.584105 (+ 116us) server_negotiation.cc:407] Connection header received
0620 23:07:41.584168 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.584169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.584178 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.584255 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:07:41.584397 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.584890 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.585858 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.586190 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.586270 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.586277 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.586414 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.586447 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:07:41.586867 5803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b861576712da4c619f23a2918337f00e" candidate_uuid: "f34cf33f244e41daaea566ceca8f6bb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088b347fe1544556a9c52207f86a4313" is_pre_election: true
I20260620 23:07:41.586944 5672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b861576712da4c619f23a2918337f00e" candidate_uuid: "f34cf33f244e41daaea566ceca8f6bb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd007fa7889f41c8b8ad629d5b455285" is_pre_election: true
I20260620 23:07:41.587020 5803 raft_consensus.cc:2454] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f34cf33f244e41daaea566ceca8f6bb0 in term 0.
I20260620 23:07:41.587096 5672 raft_consensus.cc:2454] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f34cf33f244e41daaea566ceca8f6bb0 in term 0.
I20260620 23:07:41.587282 5476 leader_election.cc:302] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 088b347fe1544556a9c52207f86a4313, f34cf33f244e41daaea566ceca8f6bb0; no voters:
I20260620 23:07:41.587416 5869 raft_consensus.cc:2781] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:07:41.587498 5869 raft_consensus.cc:482] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:07:41.587530 5869 raft_consensus.cc:3037] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:41.588200 5869 raft_consensus.cc:504] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } }
I20260620 23:07:41.588339 5869 leader_election.cc:288] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [CANDIDATE]: Term 1 election: Requested vote from peers 088b347fe1544556a9c52207f86a4313 (127.17.188.67:40555), bd007fa7889f41c8b8ad629d5b455285 (127.17.188.66:32975)
I20260620 23:07:41.588479 5672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b861576712da4c619f23a2918337f00e" candidate_uuid: "f34cf33f244e41daaea566ceca8f6bb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd007fa7889f41c8b8ad629d5b455285"
I20260620 23:07:41.588474 5803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b861576712da4c619f23a2918337f00e" candidate_uuid: "f34cf33f244e41daaea566ceca8f6bb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088b347fe1544556a9c52207f86a4313"
I20260620 23:07:41.588568 5672 raft_consensus.cc:3037] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:41.588567 5803 raft_consensus.cc:3037] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:41.589267 5672 raft_consensus.cc:2454] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f34cf33f244e41daaea566ceca8f6bb0 in term 1.
I20260620 23:07:41.589277 5803 raft_consensus.cc:2454] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f34cf33f244e41daaea566ceca8f6bb0 in term 1.
I20260620 23:07:41.589470 5474 leader_election.cc:302] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd007fa7889f41c8b8ad629d5b455285, f34cf33f244e41daaea566ceca8f6bb0; no voters:
I20260620 23:07:41.589591 5869 raft_consensus.cc:2781] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:41.589716 5869 raft_consensus.cc:686] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 1 LEADER]: Becoming Leader. State: Replica: f34cf33f244e41daaea566ceca8f6bb0, State: Running, Role: LEADER
I20260620 23:07:41.589811 5869 consensus_queue.cc:237] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } }
I20260620 23:07:41.590612 5398 catalog_manager.cc:5495] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f34cf33f244e41daaea566ceca8f6bb0 (127.17.188.65). New cstate: current_term: 1 leader_uuid: "f34cf33f244e41daaea566ceca8f6bb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f34cf33f244e41daaea566ceca8f6bb0" member_type: VOTER last_known_addr { host: "127.17.188.65" port: 43323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 } health_report { overall_health: UNKNOWN } } }
I20260620 23:07:41.613129 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.610816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49456 (local address 127.17.188.65:43323)
0620 23:07:41.610879 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:41.610881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:41.610891 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:07:41.610950 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:41.610950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:41.610970 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:41.611058 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:07:41.611203 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.611661 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.612374 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:41.612560 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:41.612628 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:41.612807 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:07:41.612906 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:07:41.612997 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:41.613015 (+ 18us) server_negotiation.cc:299] Negotiation successful
0620 23:07:41.613034 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:07:41.614811 5672 raft_consensus.cc:1261] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [term 1 FOLLOWER]: Refusing update from remote peer f34cf33f244e41daaea566ceca8f6bb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:41.614818 5803 raft_consensus.cc:1261] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [term 1 FOLLOWER]: Refusing update from remote peer f34cf33f244e41daaea566ceca8f6bb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:07:41.615150 5873 consensus_queue.cc:1035] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd007fa7889f41c8b8ad629d5b455285" member_type: VOTER last_known_addr { host: "127.17.188.66" port: 32975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:41.615281 5869 consensus_queue.cc:1035] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "088b347fe1544556a9c52207f86a4313" member_type: VOTER last_known_addr { host: "127.17.188.67" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:07:41.617480 5876 mvcc.cc:205] Tried to move back new op lower bound from 7299059145171636224 to 7299059145072484352. Current Snapshot: MvccSnapshot[applied={T|T < 7299059145171636224}]
I20260620 23:07:41.617648 5878 mvcc.cc:205] Tried to move back new op lower bound from 7299059145171636224 to 7299059145072484352. Current Snapshot: MvccSnapshot[applied={T|T < 7299059145171636224}]
I20260620 23:07:41.617686 5877 mvcc.cc:205] Tried to move back new op lower bound from 7299059145171636224 to 7299059145072484352. Current Snapshot: MvccSnapshot[applied={T|T < 7299059145171636224}]
I20260620 23:07:41.619544 5399 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:41.619629 5399 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:07:41.620318 5399 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf20f2c0a0834f0b82728c2a5159e320: Sending DeleteTablet for 3 replicas of tablet b861576712da4c619f23a2918337f00e
I20260620 23:07:41.620661 5783 tablet_service.cc:1514] Processing DeleteTablet for tablet b861576712da4c619f23a2918337f00e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:41 UTC) from {username='slave'} at 127.0.0.1:33310
I20260620 23:07:41.620692 5521 tablet_service.cc:1514] Processing DeleteTablet for tablet b861576712da4c619f23a2918337f00e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:41 UTC) from {username='slave'} at 127.0.0.1:49442
I20260620 23:07:41.620802 5652 tablet_service.cc:1514] Processing DeleteTablet for tablet b861576712da4c619f23a2918337f00e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:07:41 UTC) from {username='slave'} at 127.0.0.1:48784
I20260620 23:07:41.620904 5887 tablet_replica.cc:330] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0: stopping tablet replica
I20260620 23:07:41.620991 5887 raft_consensus.cc:2229] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:07:41.621059 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 5459
I20260620 23:07:41.621078 5887 raft_consensus.cc:2258] T b861576712da4c619f23a2918337f00e P f34cf33f244e41daaea566ceca8f6bb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:41.621207 5888 tablet_replica.cc:330] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: stopping tablet replica
I20260620 23:07:41.621285 5888 raft_consensus.cc:2229] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:41.621834 5888 raft_consensus.cc:2258] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:41.622604 5888 ts_tablet_manager.cc:1905] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:07:41.624182 5888 ts_tablet_manager.cc:1918] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:41.624243 5888 log.cc:1196] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-1/wal/wals/b861576712da4c619f23a2918337f00e
I20260620 23:07:41.624491 5888 ts_tablet_manager.cc:1939] T b861576712da4c619f23a2918337f00e P bd007fa7889f41c8b8ad629d5b455285: Deleting consensus metadata
I20260620 23:07:41.624867 5383 catalog_manager.cc:4841] TS bd007fa7889f41c8b8ad629d5b455285 (127.17.188.66:32975): tablet b861576712da4c619f23a2918337f00e (table test-table [id=09fca56bdbad4d0392a58c6e9ddb6183]) successfully deleted
I20260620 23:07:41.625375 5886 tablet_replica.cc:330] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: stopping tablet replica
I20260620 23:07:41.625483 5886 raft_consensus.cc:2229] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:07:41.625605 5886 raft_consensus.cc:2258] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:07:41.625952 5886 ts_tablet_manager.cc:1905] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 23:07:41.626616 5385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.188.65:43323 (error 108)
I20260620 23:07:41.626829 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 5590
I20260620 23:07:41.627828 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:41.627420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.65:43323 (local address 127.0.0.1:49460)
0620 23:07:41.627729 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 23:07:41.627774 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.65:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286}
I20260620 23:07:41.628063 5886 ts_tablet_manager.cc:1918] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:07:41.628155 5886 log.cc:1196] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: Deleting WAL directory at /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996599092328-18161-0/minicluster-data/ts-2/wal/wals/b861576712da4c619f23a2918337f00e
I20260620 23:07:41.628360 5886 ts_tablet_manager.cc:1939] T b861576712da4c619f23a2918337f00e P 088b347fe1544556a9c52207f86a4313: Deleting consensus metadata
W20260620 23:07:41.628401 5385 catalog_manager.cc:4584] TS f34cf33f244e41daaea566ceca8f6bb0 (127.17.188.65:43323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b861576712da4c619f23a2918337f00e: Network error: Client connection negotiation failed: client connection to 127.17.188.65:43323: connect: Connection refused (error 111)
I20260620 23:07:41.628717 5385 catalog_manager.cc:4841] TS 088b347fe1544556a9c52207f86a4313 (127.17.188.67:40555): tablet b861576712da4c619f23a2918337f00e (table test-table [id=09fca56bdbad4d0392a58c6e9ddb6183]) successfully deleted
I20260620 23:07:41.631958 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 5721
I20260620 23:07:41.636574 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 5368
2026-06-20T23:07:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (738 ms)
[----------] 6 tests from EncryptionPolicyTest (8592 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:41 dist-test-slave-zpss krb5kdc[5892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:41 dist-test-slave-zpss krb5kdc[5892](info): set up 2 sockets
Jun 20 23:07:41 dist-test-slave-zpss krb5kdc[5892](info): commencing operation
krb5kdc: starting...
W20260620 23:07:42.653445 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:42 dist-test-slave-zpss krb5kdc[5892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:42.761693 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33793
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:42.839438 5905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:42.839656 5905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:42.839705 5905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:42.840843 5905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:42.840907 5905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:42.840930 5905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:42.840952 5905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:42.840981 5905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:42.842420 5905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33793
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:42.842686 5905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:42.842922 5905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:42.847591 5911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:42.847623 5910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:42.847728 5905 server_base.cc:1034] running on GCE node
W20260620 23:07:42.847599 5913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:42.848057 5905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:42.848352 5905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:42.849512 5905 hybrid_clock.cc:648] HybridClock initialized: now 1781996862849499 us; error 43 us; skew 500 ppm
Jun 20 23:07:42 dist-test-slave-zpss krb5kdc[5892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996862, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:42.851912 5905 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:42.852321 5905 webserver.cc:460] Webserver started at http://127.0.0.1:44317/ using document root <none> and password file <none>
I20260620 23:07:42.852536 5905 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:42.852602 5905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:42.852758 5905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:42.853852 5905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "d74859bd515a4c83bb769d4b931ab865"
format_stamp: "Formatted at 2026-06-20 23:07:42 on dist-test-slave-zpss"
server_key: "2d140a23ce9a0f474171030ad7679084"
server_key_iv: "c13d4bad4b9cd6b80839f4346a50620d"
server_key_version: "encryptionkey@0"
I20260620 23:07:42.854197 5905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "d74859bd515a4c83bb769d4b931ab865"
format_stamp: "Formatted at 2026-06-20 23:07:42 on dist-test-slave-zpss"
server_key: "2d140a23ce9a0f474171030ad7679084"
server_key_iv: "c13d4bad4b9cd6b80839f4346a50620d"
server_key_version: "encryptionkey@0"
I20260620 23:07:42.855490 5905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:42.856199 5920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:42.856367 5905 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:42.856469 5905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "d74859bd515a4c83bb769d4b931ab865"
format_stamp: "Formatted at 2026-06-20 23:07:42 on dist-test-slave-zpss"
server_key: "2d140a23ce9a0f474171030ad7679084"
server_key_iv: "c13d4bad4b9cd6b80839f4346a50620d"
server_key_version: "encryptionkey@0"
I20260620 23:07:42.856539 5905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:42.875774 5905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:42.878075 5905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:42.878239 5905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:42.882025 5905 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33793
I20260620 23:07:42.882057 5972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33793 every 8 connection(s)
I20260620 23:07:42.882391 5905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:42.883199 5973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:42.885335 5973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865: Bootstrap starting.
I20260620 23:07:42.885880 5973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:42.886111 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 5905
I20260620 23:07:42.886238 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:42.886231 5973 log.cc:824] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:42.886444 18161 external_mini_cluster.cc:1428] Setting key 073e2009e4b0256d6b5b2920fd4dbaae
I20260620 23:07:42.886969 5973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865: No bootstrap required, opened a new log
I20260620 23:07:42.888952 5973 raft_consensus.cc:348] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74859bd515a4c83bb769d4b931ab865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33793 } }
I20260620 23:07:42.889124 5973 raft_consensus.cc:374] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:42.889158 5973 raft_consensus.cc:729] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74859bd515a4c83bb769d4b931ab865, State: Initialized, Role: FOLLOWER
I20260620 23:07:42.889302 5973 consensus_queue.cc:260] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74859bd515a4c83bb769d4b931ab865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33793 } }
I20260620 23:07:42.889416 5973 raft_consensus.cc:388] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:42.889456 5973 raft_consensus.cc:482] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:42.889501 5973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:42.890166 5973 raft_consensus.cc:504] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74859bd515a4c83bb769d4b931ab865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33793 } }
I20260620 23:07:42.890272 5973 leader_election.cc:302] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d74859bd515a4c83bb769d4b931ab865; no voters:
I20260620 23:07:42.890419 5973 leader_election.cc:288] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:42.890473 5978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:42.890627 5973 sys_catalog.cc:564] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:42.890638 5978 raft_consensus.cc:686] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [term 1 LEADER]: Becoming Leader. State: Replica: d74859bd515a4c83bb769d4b931ab865, State: Running, Role: LEADER
I20260620 23:07:42.890857 5978 consensus_queue.cc:237] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74859bd515a4c83bb769d4b931ab865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33793 } }
I20260620 23:07:42.891242 5979 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d74859bd515a4c83bb769d4b931ab865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74859bd515a4c83bb769d4b931ab865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33793 } } }
I20260620 23:07:42.891384 5979 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:42.891577 5980 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d74859bd515a4c83bb769d4b931ab865. Latest consensus state: current_term: 1 leader_uuid: "d74859bd515a4c83bb769d4b931ab865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74859bd515a4c83bb769d4b931ab865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33793 } } }
I20260620 23:07:42.891701 5980 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:42.891893 5987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:42.892362 5987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:42.893975 5987 catalog_manager.cc:1347] Generated new cluster ID: a91773e0b36f4d9eb85f9aa553730c93
I20260620 23:07:42.894032 5987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:07:42 dist-test-slave-zpss krb5kdc[5892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:07:42.899338 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:42.886962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49973 (local address 127.0.0.1:33793)
0620 23:07:42.887140 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:42.887148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:42.887194 (+ 46us) server_negotiation.cc:407] Connection header received
0620 23:07:42.888150 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:42.888166 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:42.888226 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:42.888368 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:42.888955 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:42.890687 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:42.891917 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:42.892488 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:42.892584 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:42.895570 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:42.895574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:42.895574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:42.895585 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:42.897582 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:42.898003 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:42.898006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:42.898007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:42.898115 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:42.898342 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:42.898343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:42.898343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:42.898654 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:42.898813 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:42.898930 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 23:07:42.899028 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
I20260620 23:07:42.900415 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:42.909613 5987 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:42.910270 5987 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:42.912743 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:42.905627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44635 (local address 127.0.0.1:33793)
0620 23:07:42.905675 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:42.905677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:42.905870 (+ 193us) server_negotiation.cc:407] Connection header received
0620 23:07:42.905897 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:42.905898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:42.905907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:42.905972 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:42.906522 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:42.907274 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:42.908230 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:42.908539 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:42.908609 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:42.910012 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:42.910014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:42.910014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:42.910021 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:42.911576 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:42.911979 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:42.911980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:42.911980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:42.912036 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:42.912253 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:42.912254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:42.912254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:42.912375 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:42.912474 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:42.912586 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 23:07:42.912650 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:07:42.916039 5987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d74859bd515a4c83bb769d4b931ab865: Generated new TSK 0
I20260620 23:07:42.916208 5987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:42.921978 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 5905
W20260620 23:07:42.926309 5890 connection.cc:535] client connection to 127.0.0.1:33793 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33793): Connection reset by peer (error 104)
2026-06-20T23:07:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:42 dist-test-slave-zpss krb5kdc[6004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:42 dist-test-slave-zpss krb5kdc[6004](info): set up 2 sockets
Jun 20 23:07:42 dist-test-slave-zpss krb5kdc[6004](info): commencing operation
krb5kdc: starting...
W20260620 23:07:43.941283 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:44 dist-test-slave-zpss krb5kdc[6004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:44.049520 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:44.127146 6017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:44.127326 6017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:44.127347 6017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:44.128418 6017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:44.128453 6017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:44.128466 6017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:44.128479 6017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:44.128491 6017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:44.129837 6017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:44.129995 6017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:44.130138 6017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:07:44.134780 6017 server_base.cc:1034] running on GCE node
W20260620 23:07:44.134758 6023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:44.134745 6022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:44.134737 6025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:44.135130 6017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:44.135396 6017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:44.136562 6017 hybrid_clock.cc:648] HybridClock initialized: now 1781996864136565 us; error 41 us; skew 500 ppm
Jun 20 23:07:44 dist-test-slave-zpss krb5kdc[6004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:44.142587 6017 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:44.143055 6017 webserver.cc:460] Webserver started at http://127.0.0.1:40821/ using document root <none> and password file <none>
I20260620 23:07:44.143280 6017 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:44.143357 6017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:44.143494 6017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:44.144416 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "29268b8902a5444393f5b6850088ad05"
format_stamp: "Formatted at 2026-06-20 23:07:44 on dist-test-slave-zpss"
server_key: "e049ea3a4ed7807d9914cb033a5c28ed"
server_key_iv: "c60fc6e2775853480e8e357ad16708db"
server_key_version: "encryptionkey@0"
I20260620 23:07:44.144755 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "29268b8902a5444393f5b6850088ad05"
format_stamp: "Formatted at 2026-06-20 23:07:44 on dist-test-slave-zpss"
server_key: "e049ea3a4ed7807d9914cb033a5c28ed"
server_key_iv: "c60fc6e2775853480e8e357ad16708db"
server_key_version: "encryptionkey@0"
I20260620 23:07:44.146039 6017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:07:44.146711 6032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:44.146876 6017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:44.146953 6017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "29268b8902a5444393f5b6850088ad05"
format_stamp: "Formatted at 2026-06-20 23:07:44 on dist-test-slave-zpss"
server_key: "e049ea3a4ed7807d9914cb033a5c28ed"
server_key_iv: "c60fc6e2775853480e8e357ad16708db"
server_key_version: "encryptionkey@0"
I20260620 23:07:44.147022 6017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:44.162550 6017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:44.164224 6017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:44.164379 6017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:44.168313 6017 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43593
I20260620 23:07:44.168341 6084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43593 every 8 connection(s)
I20260620 23:07:44.168653 6017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:44.169486 6085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:44.171572 6085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05: Bootstrap starting.
I20260620 23:07:44.172075 6085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:44.172303 6085 log.cc:824] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:44.172911 6085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05: No bootstrap required, opened a new log
I20260620 23:07:44.173502 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 6017
I20260620 23:07:44.173619 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:44.173825 18161 external_mini_cluster.cc:1428] Setting key ca63c01064fdaa57b33ee129107602c7
I20260620 23:07:44.175063 6085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29268b8902a5444393f5b6850088ad05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43593 } }
I20260620 23:07:44.175271 6085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:44.175307 6085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29268b8902a5444393f5b6850088ad05, State: Initialized, Role: FOLLOWER
I20260620 23:07:44.175419 6085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29268b8902a5444393f5b6850088ad05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43593 } }
I20260620 23:07:44.175500 6085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:44.175542 6085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:44.175595 6085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:44.176090 6085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29268b8902a5444393f5b6850088ad05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43593 } }
I20260620 23:07:44.176213 6085 leader_election.cc:302] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29268b8902a5444393f5b6850088ad05; no voters:
I20260620 23:07:44.176416 6085 leader_election.cc:288] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:44.176445 6090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:44.176555 6090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [term 1 LEADER]: Becoming Leader. State: Replica: 29268b8902a5444393f5b6850088ad05, State: Running, Role: LEADER
I20260620 23:07:44.176671 6085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:44.176705 6090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29268b8902a5444393f5b6850088ad05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43593 } }
I20260620 23:07:44.177071 6090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29268b8902a5444393f5b6850088ad05. Latest consensus state: current_term: 1 leader_uuid: "29268b8902a5444393f5b6850088ad05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29268b8902a5444393f5b6850088ad05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43593 } } }
I20260620 23:07:44.177081 6091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29268b8902a5444393f5b6850088ad05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29268b8902a5444393f5b6850088ad05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43593 } } }
I20260620 23:07:44.177206 6090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:44.177277 6091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:44.177536 6099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:44.178011 6099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:44.179857 6099 catalog_manager.cc:1347] Generated new cluster ID: 7bf3be01dc1f4779ab5943c95fd04963
I20260620 23:07:44.179913 6099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:44.180089 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:44.174310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45897 (local address 127.0.0.1:43593)
0620 23:07:44.174490 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:44.174499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:44.174533 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:07:44.175254 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:44.175262 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:44.175322 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:44.175461 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:44.175715 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:44.176737 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:44.177866 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:44.178461 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:44.178888 (+ 427us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:44.179323 (+ 435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:44.179326 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:44.179327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:44.179539 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:44.179573 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:44.179587 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:44.179673 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:44.179868 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 23:07:44.179894 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260620 23:07:44.180802 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:44.185688 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:44.182206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41607 (local address 127.0.0.1:43593)
0620 23:07:44.182240 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:44.182242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:44.182414 (+ 172us) server_negotiation.cc:407] Connection header received
0620 23:07:44.182473 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:44.182473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:44.182481 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:44.182532 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:44.182679 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:44.183354 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:44.184421 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:44.184699 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:44.184755 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:44.185434 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:44.185435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:44.185435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:44.185439 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:44.185451 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:44.185455 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:44.185513 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:44.185608 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 23:07:44.185616 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:07:44.186843 6099 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:44.187433 6099 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:44.195611 6099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29268b8902a5444393f5b6850088ad05: Generated new TSK 0
I20260620 23:07:44.195757 6099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:44.200163 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 6017
W20260620 23:07:44.204372 6002 connection.cc:535] client connection to 127.0.0.1:43593 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43593): Connection reset by peer (error 104)
2026-06-20T23:07:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1277 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:44 dist-test-slave-zpss krb5kdc[6116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:44 dist-test-slave-zpss krb5kdc[6116](info): set up 2 sockets
Jun 20 23:07:44 dist-test-slave-zpss krb5kdc[6116](info): commencing operation
krb5kdc: starting...
W20260620 23:07:45.213178 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:45 dist-test-slave-zpss krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:45.334136 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43377
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:45.409689 6129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:45.409864 6129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:45.409888 6129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:45.411006 6129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:45.411047 6129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:45.411062 6129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:45.411113 6129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:45.411132 6129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:45.412582 6129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43377
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:45.412761 6129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:45.412987 6129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:45.417505 6135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:45.417514 6134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:45.417515 6137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:45.417578 6129 server_base.cc:1034] running on GCE node
I20260620 23:07:45.417994 6129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:45.418293 6129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:45.419454 6129 hybrid_clock.cc:648] HybridClock initialized: now 1781996865419442 us; error 44 us; skew 500 ppm
Jun 20 23:07:45 dist-test-slave-zpss krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:45.421870 6129 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:45.422266 6129 webserver.cc:460] Webserver started at http://127.0.0.1:36719/ using document root <none> and password file <none>
I20260620 23:07:45.422475 6129 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:45.422544 6129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:45.422689 6129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:45.423666 6129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "5114ba27557e47d283cb97cbc8f559e4"
format_stamp: "Formatted at 2026-06-20 23:07:45 on dist-test-slave-zpss"
server_key: "865d75c838527dec28c3fe568077ef96"
server_key_iv: "56a2a3aaea1e69bcb736273648d1542c"
server_key_version: "encryptionkey@0"
I20260620 23:07:45.424014 6129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "5114ba27557e47d283cb97cbc8f559e4"
format_stamp: "Formatted at 2026-06-20 23:07:45 on dist-test-slave-zpss"
server_key: "865d75c838527dec28c3fe568077ef96"
server_key_iv: "56a2a3aaea1e69bcb736273648d1542c"
server_key_version: "encryptionkey@0"
I20260620 23:07:45.425240 6129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:45.425897 6144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:45.426048 6129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:45.426141 6129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "5114ba27557e47d283cb97cbc8f559e4"
format_stamp: "Formatted at 2026-06-20 23:07:45 on dist-test-slave-zpss"
server_key: "865d75c838527dec28c3fe568077ef96"
server_key_iv: "56a2a3aaea1e69bcb736273648d1542c"
server_key_version: "encryptionkey@0"
I20260620 23:07:45.426222 6129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:45.429168 6129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:45.431394 6129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:45.431541 6129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:45.435309 6129 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43377
I20260620 23:07:45.435389 6196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43377 every 8 connection(s)
I20260620 23:07:45.435698 6129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:45.436458 6197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:45.438200 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 6129
I20260620 23:07:45.438328 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:45.438494 18161 external_mini_cluster.cc:1428] Setting key ac775fe2127857c602e9d47caa5dc5bc
I20260620 23:07:45.438766 6197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4: Bootstrap starting.
I20260620 23:07:45.439392 6197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:45.439690 6197 log.cc:824] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:45.440321 6197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4: No bootstrap required, opened a new log
I20260620 23:07:45.440912 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:45.438973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56063 (local address 127.0.0.1:43377)
0620 23:07:45.439146 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:45.439155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:45.439184 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:07:45.439652 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:45.439670 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:45.439716 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:45.439890 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:45.440466 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:45.440470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:45.440470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:45.440474 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:45.440507 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:45.440519 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:45.440617 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:45.440654 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 23:07:45.440783 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260620 23:07:45.441854 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:45.442655 6197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5114ba27557e47d283cb97cbc8f559e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43377 } }
I20260620 23:07:45.442838 6197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:45.442873 6197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5114ba27557e47d283cb97cbc8f559e4, State: Initialized, Role: FOLLOWER
I20260620 23:07:45.442996 6197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5114ba27557e47d283cb97cbc8f559e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43377 } }
I20260620 23:07:45.443101 6197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:45.443205 6197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:45.443269 6197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:45.443989 6197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5114ba27557e47d283cb97cbc8f559e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43377 } }
I20260620 23:07:45.444116 6197 leader_election.cc:302] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5114ba27557e47d283cb97cbc8f559e4; no voters:
I20260620 23:07:45.444285 6197 leader_election.cc:288] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:45.444305 6207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:45.444416 6207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [term 1 LEADER]: Becoming Leader. State: Replica: 5114ba27557e47d283cb97cbc8f559e4, State: Running, Role: LEADER
I20260620 23:07:45.444528 6197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:45.444554 6207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5114ba27557e47d283cb97cbc8f559e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43377 } }
I20260620 23:07:45.444679 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:45.443448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48361 (local address 127.0.0.1:43377)
0620 23:07:45.443559 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:45.443561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:45.443641 (+ 80us) server_negotiation.cc:407] Connection header received
0620 23:07:45.443717 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:45.443717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:45.443721 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:45.443764 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:45.444298 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:45.444299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:45.444300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:45.444304 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:45.444319 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:45.444324 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:45.444390 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:45.444610 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 23:07:45.444622 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260620 23:07:45.444903 6208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5114ba27557e47d283cb97cbc8f559e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5114ba27557e47d283cb97cbc8f559e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43377 } } }
I20260620 23:07:45.445050 6208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:45.445025 6207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5114ba27557e47d283cb97cbc8f559e4. Latest consensus state: current_term: 1 leader_uuid: "5114ba27557e47d283cb97cbc8f559e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5114ba27557e47d283cb97cbc8f559e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43377 } } }
I20260620 23:07:45.445155 6207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:45.445400 6215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:45.446077 6215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:45.447327 6215 catalog_manager.cc:1347] Generated new cluster ID: 92fac6e64e784b5293ed5db1127561de
I20260620 23:07:45.447376 6215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:45.456743 6215 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:45.456907 6215 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:45.462181 6215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5114ba27557e47d283cb97cbc8f559e4: Generated new TSK 0
I20260620 23:07:45.462363 6215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:45.464375 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 6129
2026-06-20T23:07:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:45 dist-test-slave-zpss krb5kdc[6228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:45 dist-test-slave-zpss krb5kdc[6228](info): set up 2 sockets
Jun 20 23:07:45 dist-test-slave-zpss krb5kdc[6228](info): commencing operation
krb5kdc: starting...
W20260620 23:07:46.465196 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:46 dist-test-slave-zpss krb5kdc[6228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:46.571419 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:46.647610 6241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:46.647789 6241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:46.647810 6241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:46.648918 6241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:46.648969 6241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:46.648985 6241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:46.649037 6241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:46.649068 6241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:46.650550 6241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:46.650794 6241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:46.651031 6241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:46.655686 6247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:46.655674 6246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:46.655846 6241 server_base.cc:1034] running on GCE node
W20260620 23:07:46.655656 6249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:46.656157 6241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:46.656428 6241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:46.657583 6241 hybrid_clock.cc:648] HybridClock initialized: now 1781996866657580 us; error 39 us; skew 500 ppm
Jun 20 23:07:46 dist-test-slave-zpss krb5kdc[6228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996866, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:46.659816 6241 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:46.660197 6241 webserver.cc:460] Webserver started at http://127.0.0.1:36049/ using document root <none> and password file <none>
I20260620 23:07:46.660403 6241 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:46.660470 6241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:46.660620 6241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:46.661674 6241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "63d6042f577e45deb1d4b588f1107bfb"
format_stamp: "Formatted at 2026-06-20 23:07:46 on dist-test-slave-zpss"
server_key: "c4dce11dfe9e2c9f123833c814791dde"
server_key_iv: "523954a608d2e2306deee7f8158f9324"
server_key_version: "encryptionkey@0"
I20260620 23:07:46.662021 6241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "63d6042f577e45deb1d4b588f1107bfb"
format_stamp: "Formatted at 2026-06-20 23:07:46 on dist-test-slave-zpss"
server_key: "c4dce11dfe9e2c9f123833c814791dde"
server_key_iv: "523954a608d2e2306deee7f8158f9324"
server_key_version: "encryptionkey@0"
I20260620 23:07:46.663329 6241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:46.664031 6256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:46.664201 6241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:46.664296 6241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "63d6042f577e45deb1d4b588f1107bfb"
format_stamp: "Formatted at 2026-06-20 23:07:46 on dist-test-slave-zpss"
server_key: "c4dce11dfe9e2c9f123833c814791dde"
server_key_iv: "523954a608d2e2306deee7f8158f9324"
server_key_version: "encryptionkey@0"
I20260620 23:07:46.664369 6241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:46.670709 6241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:46.672484 6241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:46.672639 6241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:46.676404 6241 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40911
I20260620 23:07:46.676432 6308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40911 every 8 connection(s)
I20260620 23:07:46.676743 6241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:46.677529 6309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:46.679499 6309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb: Bootstrap starting.
I20260620 23:07:46.680037 6309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:46.680302 6309 log.cc:824] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:46.680922 6309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb: No bootstrap required, opened a new log
I20260620 23:07:46.682490 6309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d6042f577e45deb1d4b588f1107bfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40911 } }
I20260620 23:07:46.682610 6309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:46.682664 6309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d6042f577e45deb1d4b588f1107bfb, State: Initialized, Role: FOLLOWER
I20260620 23:07:46.682842 6309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d6042f577e45deb1d4b588f1107bfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40911 } }
I20260620 23:07:46.682904 6309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:46.682942 6309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:46.682991 6309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:46.683465 6309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d6042f577e45deb1d4b588f1107bfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40911 } }
I20260620 23:07:46.683568 6309 leader_election.cc:302] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63d6042f577e45deb1d4b588f1107bfb; no voters:
I20260620 23:07:46.683740 6309 leader_election.cc:288] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:46.683768 6312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:46.683959 6312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [term 1 LEADER]: Becoming Leader. State: Replica: 63d6042f577e45deb1d4b588f1107bfb, State: Running, Role: LEADER
I20260620 23:07:46.683998 6309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:46.684063 6312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d6042f577e45deb1d4b588f1107bfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40911 } }
I20260620 23:07:46.684437 6314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63d6042f577e45deb1d4b588f1107bfb. Latest consensus state: current_term: 1 leader_uuid: "63d6042f577e45deb1d4b588f1107bfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d6042f577e45deb1d4b588f1107bfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40911 } } }
I20260620 23:07:46.684548 6313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63d6042f577e45deb1d4b588f1107bfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d6042f577e45deb1d4b588f1107bfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40911 } } }
I20260620 23:07:46.684577 6314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:46.684648 6313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:46.685034 6321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:46.685559 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 6241
I20260620 23:07:46.685583 6321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:46.685658 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:46.685859 18161 external_mini_cluster.cc:1428] Setting key eef6cb37d4b406b5381219e23e5337f4
I20260620 23:07:46.689747 6321 catalog_manager.cc:1347] Generated new cluster ID: e1c80ed0d0ec40ab88089a397378bf6f
I20260620 23:07:46.689801 6321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:07:46 dist-test-slave-zpss krb5kdc[6228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:07:46.695937 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:46.686496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56957 (local address 127.0.0.1:40911)
0620 23:07:46.686633 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:46.686640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:46.686668 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:07:46.686766 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:46.686772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:46.686830 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:46.686968 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:46.687533 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:46.688847 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:46.689891 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:46.690475 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:46.690588 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:46.692833 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:46.692837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:46.692837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:46.692847 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:46.694456 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:46.694830 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:46.694833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:46.694833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:46.694929 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:46.695139 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:46.695140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:46.695140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:46.695349 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:46.695514 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:46.695617 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 23:07:46.695750 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":52,"threads_started":1}
I20260620 23:07:46.696862 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:46.703574 6321 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:46.704192 6321 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:46.707569 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:46.698265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46023 (local address 127.0.0.1:40911)
0620 23:07:46.698297 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:46.698299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:46.701473 (+ 3174us) server_negotiation.cc:407] Connection header received
0620 23:07:46.701543 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:46.701543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:46.701553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:46.701616 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:46.702110 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:46.702580 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:46.703448 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:46.703756 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:46.703828 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:46.705053 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:46.705055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:46.705055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:46.705061 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:46.706557 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:46.706901 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:46.706902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:46.706902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:46.706955 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:46.707123 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:46.707124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:46.707124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:46.707241 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:46.707323 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:46.707418 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 23:07:46.707492 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":17}
I20260620 23:07:46.715310 6321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63d6042f577e45deb1d4b588f1107bfb: Generated new TSK 0
I20260620 23:07:46.715471 6321 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:46.719485 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 6241
2026-06-20T23:07:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1255 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:46 dist-test-slave-zpss krb5kdc[6340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:46 dist-test-slave-zpss krb5kdc[6340](info): set up 2 sockets
Jun 20 23:07:46 dist-test-slave-zpss krb5kdc[6340](info): commencing operation
krb5kdc: starting...
W20260620 23:07:47.737416 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:47 dist-test-slave-zpss krb5kdc[6340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:47.846336 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33447
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:47.923146 6353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:47.923357 6353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:47.923395 6353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:47.924491 6353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:47.924556 6353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:47.924579 6353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:47.924607 6353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:47.924643 6353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:47.926122 6353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:47.926400 6353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:47.926608 6353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:47.931348 6359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:47.931365 6361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:47.931409 6353 server_base.cc:1034] running on GCE node
W20260620 23:07:47.931445 6358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:47.931749 6353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:47.932044 6353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:47.933214 6353 hybrid_clock.cc:648] HybridClock initialized: now 1781996867933207 us; error 45 us; skew 500 ppm
Jun 20 23:07:47 dist-test-slave-zpss krb5kdc[6340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996867, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:47.935509 6353 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:47.935843 6353 webserver.cc:460] Webserver started at http://127.0.0.1:39679/ using document root <none> and password file <none>
I20260620 23:07:47.936021 6353 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:47.936057 6353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:47.936148 6353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:47.937085 6353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "bf22cd1be772407793d90b9f44866f77"
format_stamp: "Formatted at 2026-06-20 23:07:47 on dist-test-slave-zpss"
server_key: "4c5898f683c55d9b69d262a519cb9444"
server_key_iv: "353e212f127265faa05bf7b509c41e9a"
server_key_version: "encryptionkey@0"
I20260620 23:07:47.937438 6353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "bf22cd1be772407793d90b9f44866f77"
format_stamp: "Formatted at 2026-06-20 23:07:47 on dist-test-slave-zpss"
server_key: "4c5898f683c55d9b69d262a519cb9444"
server_key_iv: "353e212f127265faa05bf7b509c41e9a"
server_key_version: "encryptionkey@0"
I20260620 23:07:47.938634 6353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:07:47.939208 6368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:47.939356 6353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:47.939435 6353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "bf22cd1be772407793d90b9f44866f77"
format_stamp: "Formatted at 2026-06-20 23:07:47 on dist-test-slave-zpss"
server_key: "4c5898f683c55d9b69d262a519cb9444"
server_key_iv: "353e212f127265faa05bf7b509c41e9a"
server_key_version: "encryptionkey@0"
I20260620 23:07:47.939499 6353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:47.951793 6353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:47.953506 6353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:47.953616 6353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:47.957046 6353 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33447
I20260620 23:07:47.957077 6420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33447 every 8 connection(s)
I20260620 23:07:47.957466 6353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:47.958308 6421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:47.960376 6421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77: Bootstrap starting.
I20260620 23:07:47.960855 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 6353
I20260620 23:07:47.960963 6421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:47.960975 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:47.961159 18161 external_mini_cluster.cc:1428] Setting key 6672b2dca9ef77b143f8488f33e1be6e
I20260620 23:07:47.961297 6421 log.cc:824] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:47.962031 6421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77: No bootstrap required, opened a new log
I20260620 23:07:47.963820 6421 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf22cd1be772407793d90b9f44866f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } }
I20260620 23:07:47.963974 6421 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:47.964020 6421 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf22cd1be772407793d90b9f44866f77, State: Initialized, Role: FOLLOWER
I20260620 23:07:47.964136 6421 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf22cd1be772407793d90b9f44866f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } }
I20260620 23:07:47.964203 6421 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:47.964223 6421 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:47.964283 6421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:47.964768 6421 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf22cd1be772407793d90b9f44866f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } }
I20260620 23:07:47.964880 6421 leader_election.cc:302] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf22cd1be772407793d90b9f44866f77; no voters:
I20260620 23:07:47.965029 6421 leader_election.cc:288] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:47.965050 6426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:47.965161 6426 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [term 1 LEADER]: Becoming Leader. State: Replica: bf22cd1be772407793d90b9f44866f77, State: Running, Role: LEADER
I20260620 23:07:47.965235 6426 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf22cd1be772407793d90b9f44866f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } }
I20260620 23:07:47.965286 6421 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:47.965571 6428 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf22cd1be772407793d90b9f44866f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf22cd1be772407793d90b9f44866f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } } }
I20260620 23:07:47.965682 6428 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:47.965862 6427 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf22cd1be772407793d90b9f44866f77. Latest consensus state: current_term: 1 leader_uuid: "bf22cd1be772407793d90b9f44866f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf22cd1be772407793d90b9f44866f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } } }
I20260620 23:07:47.965927 6427 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:47.966018 6433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:47.966480 6433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:47.966809 6425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:47.961628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43663 (local address 127.0.0.1:33447)
0620 23:07:47.961791 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:47.961801 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:47.961838 (+ 37us) server_negotiation.cc:407] Connection header received
0620 23:07:47.962325 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:47.962332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:47.962375 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:47.962529 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:47.962786 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:47.963628 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:47.964625 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:47.965124 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:47.965233 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:47.966089 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:47.966092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:47.966093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:47.966272 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:47.966299 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:47.966309 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:47.966387 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:47.966472 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 23:07:47.966531 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260620 23:07:47.967765 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:47.969425 6433 catalog_manager.cc:1347] Generated new cluster ID: 86faf90ac0854b6faec4d0b454e28b37
I20260620 23:07:47.969522 6433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:47.972556 6425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:47.969182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56997 (local address 127.0.0.1:33447)
0620 23:07:47.969219 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:47.969221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:47.969353 (+ 132us) server_negotiation.cc:407] Connection header received
0620 23:07:47.969400 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:47.969400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:47.969409 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:47.969467 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:47.969623 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:47.970319 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:47.971340 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:47.971627 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:47.971682 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:47.972336 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:47.972337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:47.972337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:47.972341 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:47.972354 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:47.972358 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:47.972407 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:47.972476 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:07:47.972493 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:07:47.981163 6433 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:47.981689 6433 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:47.987920 6433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf22cd1be772407793d90b9f44866f77: Generated new TSK 0
I20260620 23:07:47.988095 6433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:47.993753 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 6353
2026-06-20T23:07:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:48 dist-test-slave-zpss krb5kdc[6452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:48 dist-test-slave-zpss krb5kdc[6452](info): set up 2 sockets
Jun 20 23:07:48 dist-test-slave-zpss krb5kdc[6452](info): commencing operation
krb5kdc: starting...
W20260620 23:07:49.009166 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:49 dist-test-slave-zpss krb5kdc[6452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:49.115921 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:49.192878 6465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:49.193058 6465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:49.193079 6465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:49.194224 6465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:49.194264 6465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:49.194278 6465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:49.194290 6465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:49.194337 6465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:49.195688 6465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:49.195868 6465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:49.196081 6465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:49.200773 6473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:49.200783 6470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:49.200773 6471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:49.201148 6465 server_base.cc:1034] running on GCE node
I20260620 23:07:49.201424 6465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:49.201707 6465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:49.202867 6465 hybrid_clock.cc:648] HybridClock initialized: now 1781996869202845 us; error 52 us; skew 500 ppm
Jun 20 23:07:49 dist-test-slave-zpss krb5kdc[6452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996869, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:49.207947 6465 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:49.208382 6465 webserver.cc:460] Webserver started at http://127.0.0.1:43725/ using document root <none> and password file <none>
I20260620 23:07:49.208590 6465 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:49.208657 6465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:49.208798 6465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:49.209846 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "6957e68b14ce4d06b66c94483009198a"
format_stamp: "Formatted at 2026-06-20 23:07:49 on dist-test-slave-zpss"
server_key: "08084403d0160ec63b97448cf2a9e81e"
server_key_iv: "9a4c6dd26c7ee176c061342b65f40aad"
server_key_version: "encryptionkey@0"
I20260620 23:07:49.210191 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "6957e68b14ce4d06b66c94483009198a"
format_stamp: "Formatted at 2026-06-20 23:07:49 on dist-test-slave-zpss"
server_key: "08084403d0160ec63b97448cf2a9e81e"
server_key_iv: "9a4c6dd26c7ee176c061342b65f40aad"
server_key_version: "encryptionkey@0"
I20260620 23:07:49.211428 6465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:49.212114 6480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:49.212280 6465 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:49.212381 6465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "6957e68b14ce4d06b66c94483009198a"
format_stamp: "Formatted at 2026-06-20 23:07:49 on dist-test-slave-zpss"
server_key: "08084403d0160ec63b97448cf2a9e81e"
server_key_iv: "9a4c6dd26c7ee176c061342b65f40aad"
server_key_version: "encryptionkey@0"
I20260620 23:07:49.212451 6465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:49.215685 6465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:49.217808 6465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:49.217909 6465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:49.221589 6465 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42271
I20260620 23:07:49.221642 6532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42271 every 8 connection(s)
I20260620 23:07:49.221935 6465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:49.222709 6533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:49.224743 6533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a: Bootstrap starting.
I20260620 23:07:49.225241 6533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:49.225543 6533 log.cc:824] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:49.226136 6533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a: No bootstrap required, opened a new log
I20260620 23:07:49.227614 6533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6957e68b14ce4d06b66c94483009198a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } }
I20260620 23:07:49.227748 6533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:49.227771 6533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6957e68b14ce4d06b66c94483009198a, State: Initialized, Role: FOLLOWER
I20260620 23:07:49.227881 6533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6957e68b14ce4d06b66c94483009198a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } }
I20260620 23:07:49.227927 6533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:49.227947 6533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:49.227977 6533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:49.228421 6533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6957e68b14ce4d06b66c94483009198a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } }
I20260620 23:07:49.228498 6533 leader_election.cc:302] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6957e68b14ce4d06b66c94483009198a; no voters:
I20260620 23:07:49.228607 6533 leader_election.cc:288] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:49.228652 6536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:49.228802 6536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [term 1 LEADER]: Becoming Leader. State: Replica: 6957e68b14ce4d06b66c94483009198a, State: Running, Role: LEADER
I20260620 23:07:49.228852 6533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:49.228921 6536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6957e68b14ce4d06b66c94483009198a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } }
I20260620 23:07:49.229282 6537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6957e68b14ce4d06b66c94483009198a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6957e68b14ce4d06b66c94483009198a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } } }
I20260620 23:07:49.229419 6537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:49.229286 6538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6957e68b14ce4d06b66c94483009198a. Latest consensus state: current_term: 1 leader_uuid: "6957e68b14ce4d06b66c94483009198a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6957e68b14ce4d06b66c94483009198a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42271 } } }
I20260620 23:07:49.229573 6538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:49.229990 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 6465
I20260620 23:07:49.230095 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:49.230348 18161 external_mini_cluster.cc:1428] Setting key 22226e29fa3c24ec11bd6ea6d883c234
I20260620 23:07:49.230556 6540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:49.231176 6540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:49.232492 6540 catalog_manager.cc:1347] Generated new cluster ID: edd086ca17854dcbb2223948abc3c0b6
I20260620 23:07:49.232551 6540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:49.233474 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:49.230776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55459 (local address 127.0.0.1:42271)
0620 23:07:49.231704 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:49.231721 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:49.231745 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:07:49.231829 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:49.231834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:49.231856 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:49.232001 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:49.232971 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:49.232974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:49.232974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:49.232979 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:49.233014 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:49.233028 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:49.233079 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:49.233241 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 23:07:49.233282 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":878,"thread_start_us":63,"threads_started":1}
I20260620 23:07:49.234565 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:49.237005 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:49.235982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41779 (local address 127.0.0.1:42271)
0620 23:07:49.236010 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:49.236012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:49.236161 (+ 149us) server_negotiation.cc:407] Connection header received
0620 23:07:49.236191 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:49.236192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:49.236196 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:49.236261 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:49.236797 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:49.236798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:49.236798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:49.236802 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:49.236815 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:49.236819 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:49.236880 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:49.236933 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 23:07:49.236945 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:07:49.261245 6540 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:49.261430 6540 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:49.265645 6540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6957e68b14ce4d06b66c94483009198a: Generated new TSK 0
I20260620 23:07:49.265808 6540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:49.269130 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 6465
2026-06-20T23:07:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:49 dist-test-slave-zpss krb5kdc[6564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:49 dist-test-slave-zpss krb5kdc[6564](info): set up 2 sockets
Jun 20 23:07:49 dist-test-slave-zpss krb5kdc[6564](info): commencing operation
krb5kdc: starting...
W20260620 23:07:50.285279 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:50 dist-test-slave-zpss krb5kdc[6564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:50.392019 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:50.465083 6577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:50.465250 6577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:50.465271 6577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:50.466358 6577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:50.466398 6577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:50.466410 6577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:50.466423 6577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:50.466434 6577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:50.467725 6577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:50.467895 6577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:50.468037 6577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:50.472615 6583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:50.472621 6585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:50.472621 6582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:50.473392 6577 server_base.cc:1034] running on GCE node
I20260620 23:07:50.473591 6577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:50.473826 6577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:50.474960 6577 hybrid_clock.cc:648] HybridClock initialized: now 1781996870474952 us; error 30 us; skew 500 ppm
Jun 20 23:07:50 dist-test-slave-zpss krb5kdc[6564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996870, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:50.477403 6577 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:50.477784 6577 webserver.cc:460] Webserver started at http://127.0.0.1:41765/ using document root <none> and password file <none>
I20260620 23:07:50.478006 6577 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:50.478080 6577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:50.478225 6577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:50.479131 6577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "5bc8297e13934dfeb02e8a4fdb43c044"
format_stamp: "Formatted at 2026-06-20 23:07:50 on dist-test-slave-zpss"
server_key: "7e40cd5430b71b0f856f74082f77382d"
server_key_iv: "4d4d80fb1d65a629e9f7906f66b80eeb"
server_key_version: "encryptionkey@0"
I20260620 23:07:50.479447 6577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "5bc8297e13934dfeb02e8a4fdb43c044"
format_stamp: "Formatted at 2026-06-20 23:07:50 on dist-test-slave-zpss"
server_key: "7e40cd5430b71b0f856f74082f77382d"
server_key_iv: "4d4d80fb1d65a629e9f7906f66b80eeb"
server_key_version: "encryptionkey@0"
I20260620 23:07:50.480664 6577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:07:50.481361 6592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:50.481535 6577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:50.481635 6577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "5bc8297e13934dfeb02e8a4fdb43c044"
format_stamp: "Formatted at 2026-06-20 23:07:50 on dist-test-slave-zpss"
server_key: "7e40cd5430b71b0f856f74082f77382d"
server_key_iv: "4d4d80fb1d65a629e9f7906f66b80eeb"
server_key_version: "encryptionkey@0"
I20260620 23:07:50.481717 6577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:50.504810 6577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:50.507063 6577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:50.507212 6577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:50.510857 6577 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41067
I20260620 23:07:50.510890 6644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41067 every 8 connection(s)
I20260620 23:07:50.511250 6577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:50.512065 6645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:50.514240 6645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044: Bootstrap starting.
I20260620 23:07:50.514773 6645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:50.515040 6645 log.cc:824] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:50.515674 6645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044: No bootstrap required, opened a new log
I20260620 23:07:50.516396 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 6577
I20260620 23:07:50.516505 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:50.516696 18161 external_mini_cluster.cc:1428] Setting key 546ae77e1a9d3125af455e22055d1207
I20260620 23:07:50.517558 6645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc8297e13934dfeb02e8a4fdb43c044" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } }
I20260620 23:07:50.517750 6645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:50.517792 6645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bc8297e13934dfeb02e8a4fdb43c044, State: Initialized, Role: FOLLOWER
I20260620 23:07:50.517977 6645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc8297e13934dfeb02e8a4fdb43c044" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } }
I20260620 23:07:50.518066 6645 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:50.518100 6645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:50.518153 6645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:50.519042 6645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc8297e13934dfeb02e8a4fdb43c044" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } }
I20260620 23:07:50.519129 6645 leader_election.cc:302] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bc8297e13934dfeb02e8a4fdb43c044; no voters:
I20260620 23:07:50.519345 6645 leader_election.cc:288] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:50.519366 6650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:50.519472 6650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [term 1 LEADER]: Becoming Leader. State: Replica: 5bc8297e13934dfeb02e8a4fdb43c044, State: Running, Role: LEADER
I20260620 23:07:50.519610 6650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc8297e13934dfeb02e8a4fdb43c044" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } }
I20260620 23:07:50.519610 6645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:50.520005 6651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bc8297e13934dfeb02e8a4fdb43c044" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc8297e13934dfeb02e8a4fdb43c044" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } } }
I20260620 23:07:50.520226 6651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:50.520151 6652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bc8297e13934dfeb02e8a4fdb43c044. Latest consensus state: current_term: 1 leader_uuid: "5bc8297e13934dfeb02e8a4fdb43c044" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc8297e13934dfeb02e8a4fdb43c044" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } } }
I20260620 23:07:50.520452 6652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:50.520668 6654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:50.521198 6654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:50.522918 6654 catalog_manager.cc:1347] Generated new cluster ID: 97adbeabad9e401d8f1f7ddc3af94fe8
I20260620 23:07:50.522971 6654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:07:50 dist-test-slave-zpss krb5kdc[6564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:07:50.527138 6649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:50.517195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:51937 (local address 127.0.0.1:41067)
0620 23:07:50.517399 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:50.517407 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:50.517431 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:07:50.517518 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:50.517523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:50.517568 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:50.517672 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:50.518534 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:50.519574 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:50.520670 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:50.521360 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:50.521814 (+ 454us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:50.523966 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:50.523969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:50.523970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:50.523981 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:50.525638 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:50.526002 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:50.526006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:50.526006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:50.526122 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:50.526353 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:50.526354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:50.526354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:50.526573 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:50.526800 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:50.526875 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:07:50.526978 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260620 23:07:50.527944 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:50.530648 6654 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:50.531286 6654 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:50.534694 6654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bc8297e13934dfeb02e8a4fdb43c044: Generated new TSK 0
I20260620 23:07:50.534848 6654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:50.535400 6649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:50.529341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:37125 (local address 127.0.0.1:41067)
0620 23:07:50.529388 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:50.529389 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:50.529488 (+ 99us) server_negotiation.cc:407] Connection header received
0620 23:07:50.529538 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:50.529539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:50.529548 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:50.529596 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:50.530055 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:50.530666 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:50.531415 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:50.531623 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:50.531652 (+ 29us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:50.532784 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:50.532786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:50.532786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:50.532793 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:50.534189 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:50.534589 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:50.534590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:50.534590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:50.534646 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:50.534883 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:50.534884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:50.534885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:50.534997 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:50.535098 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:50.535252 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 23:07:50.535319 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:07:50.537153 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 6577
W20260620 23:07:50.541206 6562 connection.cc:535] client connection to 127.0.0.1:41067 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41067): Connection reset by peer (error 104)
2026-06-20T23:07:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:50 dist-test-slave-zpss krb5kdc[6676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:50 dist-test-slave-zpss krb5kdc[6676](info): set up 2 sockets
Jun 20 23:07:50 dist-test-slave-zpss krb5kdc[6676](info): commencing operation
krb5kdc: starting...
W20260620 23:07:51.549175 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:51 dist-test-slave-zpss krb5kdc[6676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:51.655792 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:51.732470 6689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:51.732692 6689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:51.732729 6689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:51.733978 6689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:51.734066 6689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:51.734093 6689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:51.734124 6689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:51.734165 6689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:51.735600 6689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:51.735867 6689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:51.736083 6689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:51.740585 6694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:51.740592 6695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:51.740725 6689 server_base.cc:1034] running on GCE node
W20260620 23:07:51.740592 6697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:51.741094 6689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:51.741426 6689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:51.742597 6689 hybrid_clock.cc:648] HybridClock initialized: now 1781996871742590 us; error 36 us; skew 500 ppm
Jun 20 23:07:51 dist-test-slave-zpss krb5kdc[6676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:51.744993 6689 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:51.745448 6689 webserver.cc:460] Webserver started at http://127.0.0.1:38985/ using document root <none> and password file <none>
I20260620 23:07:51.745671 6689 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:51.745736 6689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:51.745864 6689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:51.746835 6689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0"
format_stamp: "Formatted at 2026-06-20 23:07:51 on dist-test-slave-zpss"
server_key: "cd6cc3dd2ee2f15fa901eb6092567cfa"
server_key_iv: "bedb3c1c5495f630c6847f2217a4510c"
server_key_version: "encryptionkey@0"
I20260620 23:07:51.747169 6689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0"
format_stamp: "Formatted at 2026-06-20 23:07:51 on dist-test-slave-zpss"
server_key: "cd6cc3dd2ee2f15fa901eb6092567cfa"
server_key_iv: "bedb3c1c5495f630c6847f2217a4510c"
server_key_version: "encryptionkey@0"
I20260620 23:07:51.748400 6689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:07:51.749181 6704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:51.749356 6689 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:51.749475 6689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0"
format_stamp: "Formatted at 2026-06-20 23:07:51 on dist-test-slave-zpss"
server_key: "cd6cc3dd2ee2f15fa901eb6092567cfa"
server_key_iv: "bedb3c1c5495f630c6847f2217a4510c"
server_key_version: "encryptionkey@0"
I20260620 23:07:51.749557 6689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:51.765936 6689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:51.768230 6689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:51.768419 6689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:51.772276 6689 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44611
I20260620 23:07:51.772320 6756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44611 every 8 connection(s)
I20260620 23:07:51.772684 6689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:51.773533 6757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:51.775606 6757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0: Bootstrap starting.
I20260620 23:07:51.776168 6757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:51.776448 6757 log.cc:824] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:51.777139 6757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0: No bootstrap required, opened a new log
I20260620 23:07:51.778707 6757 raft_consensus.cc:348] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260620 23:07:51.778862 6757 raft_consensus.cc:374] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:51.778884 6757 raft_consensus.cc:729] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26fb4f3035c41feaaaa2a96abd4fbc0, State: Initialized, Role: FOLLOWER
I20260620 23:07:51.779054 6757 consensus_queue.cc:260] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260620 23:07:51.779115 6757 raft_consensus.cc:388] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:51.779135 6757 raft_consensus.cc:482] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:51.779206 6757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:51.779702 6757 raft_consensus.cc:504] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260620 23:07:51.779810 6757 leader_election.cc:302] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a26fb4f3035c41feaaaa2a96abd4fbc0; no voters:
I20260620 23:07:51.779995 6757 leader_election.cc:288] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:51.780017 6760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:51.780030 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 6689
I20260620 23:07:51.780153 6760 raft_consensus.cc:686] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [term 1 LEADER]: Becoming Leader. State: Replica: a26fb4f3035c41feaaaa2a96abd4fbc0, State: Running, Role: LEADER
I20260620 23:07:51.780157 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:51.780241 6757 sys_catalog.cc:564] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:51.780223 6760 consensus_queue.cc:237] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260620 23:07:51.780364 18161 external_mini_cluster.cc:1428] Setting key e746e9f704c8db75832bc14ab87c56d0
I20260620 23:07:51.780652 6762 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260620 23:07:51.780752 6762 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:51.780666 6761 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a26fb4f3035c41feaaaa2a96abd4fbc0. Latest consensus state: current_term: 1 leader_uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26fb4f3035c41feaaaa2a96abd4fbc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260620 23:07:51.781001 6761 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:51.781708 6769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:51.782235 6769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:51.783993 6769 catalog_manager.cc:1347] Generated new cluster ID: d7f72c47a0ea4e108beb657d29e77400
I20260620 23:07:51.784042 6769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:51.786650 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:51.780801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:49807 (local address 127.0.0.1:44611)
0620 23:07:51.781149 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:51.781158 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:51.781189 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:07:51.781297 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:51.781304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:51.781382 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:51.781519 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:51.782074 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:51.783503 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:51.784569 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:51.785170 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:51.785283 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:51.785964 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:51.785967 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:51.785968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:51.786167 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:51.786205 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:51.786221 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:51.786309 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:51.786387 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 23:07:51.786414 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":79,"threads_started":1}
I20260620 23:07:51.787518 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:51.792613 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:51.789169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:37109 (local address 127.0.0.1:44611)
0620 23:07:51.789205 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:51.789206 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:51.789276 (+ 70us) server_negotiation.cc:407] Connection header received
0620 23:07:51.789384 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:51.789385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:51.789394 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:51.789447 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:51.789610 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:51.790334 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:51.791356 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:51.791647 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:51.791706 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:51.792381 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:51.792382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:51.792382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:51.792386 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:51.792398 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:51.792402 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:51.792452 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:51.792538 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 23:07:51.792549 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:07:51.794831 6769 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:51.795287 6769 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:51.802009 6769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a26fb4f3035c41feaaaa2a96abd4fbc0: Generated new TSK 0
I20260620 23:07:51.802191 6769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:51.808213 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 6689
W20260620 23:07:51.812412 6674 connection.cc:535] client connection to 127.0.0.1:44611 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44611): Connection reset by peer (error 104)
2026-06-20T23:07:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1271 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:51 dist-test-slave-zpss krb5kdc[6788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:51 dist-test-slave-zpss krb5kdc[6788](info): set up 2 sockets
Jun 20 23:07:51 dist-test-slave-zpss krb5kdc[6788](info): commencing operation
krb5kdc: starting...
W20260620 23:07:52.825246 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:52 dist-test-slave-zpss krb5kdc[6788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:52.935616 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39643
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:53.015170 6801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:53.015369 6801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:53.015405 6801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:53.016422 6801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:53.016480 6801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:53.016505 6801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:53.016536 6801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:53.016564 6801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:53.018327 6801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:53.018615 6801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:53.018850 6801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:53.024006 6806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:53.024014 6807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:53.024161 6801 server_base.cc:1034] running on GCE node
W20260620 23:07:53.024044 6809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:53.024411 6801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:53.024678 6801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:53.025815 6801 hybrid_clock.cc:648] HybridClock initialized: now 1781996873025803 us; error 28 us; skew 500 ppm
Jun 20 23:07:53 dist-test-slave-zpss krb5kdc[6788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996873, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:53.028105 6801 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:53.028551 6801 webserver.cc:460] Webserver started at http://127.0.0.1:43261/ using document root <none> and password file <none>
I20260620 23:07:53.028769 6801 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:53.028832 6801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:53.028973 6801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:53.030110 6801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "f5aa882dfebf413b863c30ea85c2b1f5"
format_stamp: "Formatted at 2026-06-20 23:07:53 on dist-test-slave-zpss"
server_key: "b0242a5cfbdc72522003e664a81d641a"
server_key_iv: "11ac588bcbd347c118e0c7e280337c6c"
server_key_version: "encryptionkey@0"
I20260620 23:07:53.030467 6801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "f5aa882dfebf413b863c30ea85c2b1f5"
format_stamp: "Formatted at 2026-06-20 23:07:53 on dist-test-slave-zpss"
server_key: "b0242a5cfbdc72522003e664a81d641a"
server_key_iv: "11ac588bcbd347c118e0c7e280337c6c"
server_key_version: "encryptionkey@0"
I20260620 23:07:53.031728 6801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:53.032399 6816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:53.032560 6801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:53.032655 6801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "f5aa882dfebf413b863c30ea85c2b1f5"
format_stamp: "Formatted at 2026-06-20 23:07:53 on dist-test-slave-zpss"
server_key: "b0242a5cfbdc72522003e664a81d641a"
server_key_iv: "11ac588bcbd347c118e0c7e280337c6c"
server_key_version: "encryptionkey@0"
I20260620 23:07:53.032732 6801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:53.036000 6801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:53.037799 6801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:53.037974 6801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:53.041909 6801 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39643
I20260620 23:07:53.041944 6868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39643 every 8 connection(s)
I20260620 23:07:53.042241 6801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:53.043008 6869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:53.045068 6869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5: Bootstrap starting.
I20260620 23:07:53.045668 6869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:53.045931 6869 log.cc:824] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:53.046535 6869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5: No bootstrap required, opened a new log
I20260620 23:07:53.048024 6869 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aa882dfebf413b863c30ea85c2b1f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39643 } }
I20260620 23:07:53.048156 6869 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:53.048200 6869 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5aa882dfebf413b863c30ea85c2b1f5, State: Initialized, Role: FOLLOWER
I20260620 23:07:53.048343 6869 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aa882dfebf413b863c30ea85c2b1f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39643 } }
I20260620 23:07:53.048419 6869 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:53.048468 6869 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:53.048516 6869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:53.049010 6869 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aa882dfebf413b863c30ea85c2b1f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39643 } }
I20260620 23:07:53.049119 6869 leader_election.cc:302] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5aa882dfebf413b863c30ea85c2b1f5; no voters:
I20260620 23:07:53.049291 6869 leader_election.cc:288] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:53.049358 6872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:53.049541 6872 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [term 1 LEADER]: Becoming Leader. State: Replica: f5aa882dfebf413b863c30ea85c2b1f5, State: Running, Role: LEADER
I20260620 23:07:53.049602 6869 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:53.049623 6872 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aa882dfebf413b863c30ea85c2b1f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39643 } }
I20260620 23:07:53.049876 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 6801
I20260620 23:07:53.049991 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:53.050100 6873 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5aa882dfebf413b863c30ea85c2b1f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aa882dfebf413b863c30ea85c2b1f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39643 } } }
I20260620 23:07:53.050175 6873 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:53.050200 18161 external_mini_cluster.cc:1428] Setting key 9a0e0076d1f658780a29cc4e82374e30
I20260620 23:07:53.050359 6880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:53.050853 6880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:53.051071 6874 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5aa882dfebf413b863c30ea85c2b1f5. Latest consensus state: current_term: 1 leader_uuid: "f5aa882dfebf413b863c30ea85c2b1f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5aa882dfebf413b863c30ea85c2b1f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39643 } } }
I20260620 23:07:53.051252 6874 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:53.052469 6880 catalog_manager.cc:1347] Generated new cluster ID: 6208a97c560644a196cc7d99791a197a
I20260620 23:07:53.052516 6880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:53.053108 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:53.050856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:34471 (local address 127.0.0.1:39643)
0620 23:07:53.051036 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:53.051046 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:53.051365 (+ 319us) server_negotiation.cc:407] Connection header received
0620 23:07:53.051499 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:53.051504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:53.051550 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:53.051693 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:53.052672 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:53.052675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:53.052675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:53.052679 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:53.052723 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:53.052739 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:53.052813 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:53.052882 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:07:53.052927 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":61,"threads_started":1}
I20260620 23:07:53.054068 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:53.061506 6880 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:53.061650 6880 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:53.065819 6880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5aa882dfebf413b863c30ea85c2b1f5: Generated new TSK 0
I20260620 23:07:53.065974 6880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:53.066596 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:53.057593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:32919 (local address 127.0.0.1:39643)
0620 23:07:53.057625 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:53.057627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:53.065506 (+ 7879us) server_negotiation.cc:407] Connection header received
0620 23:07:53.065565 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:53.065566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:53.065572 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:53.065638 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:53.066363 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:53.066364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:53.066364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:53.066368 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:53.066381 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:53.066385 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:53.066430 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:53.066483 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 23:07:53.066494 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:07:53.068087 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 6801
2026-06-20T23:07:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:53 dist-test-slave-zpss krb5kdc[6900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:53 dist-test-slave-zpss krb5kdc[6900](info): set up 2 sockets
Jun 20 23:07:53 dist-test-slave-zpss krb5kdc[6900](info): commencing operation
krb5kdc: starting...
W20260620 23:07:54.089538 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:54 dist-test-slave-zpss krb5kdc[6900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:54.199986 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:54.278477 6913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:54.278663 6913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:54.278687 6913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:54.279791 6913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:54.279834 6913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:54.279850 6913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:54.279899 6913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:54.279919 6913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:54.281469 6913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:54.281697 6913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:54.281908 6913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:54.286571 6919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:54.286586 6918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:54.286590 6921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:54.286763 6913 server_base.cc:1034] running on GCE node
I20260620 23:07:54.286965 6913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:54.287225 6913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:54.288359 6913 hybrid_clock.cc:648] HybridClock initialized: now 1781996874288348 us; error 28 us; skew 500 ppm
Jun 20 23:07:54 dist-test-slave-zpss krb5kdc[6900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:54.295887 6913 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:54.296280 6913 webserver.cc:460] Webserver started at http://127.0.0.1:33337/ using document root <none> and password file <none>
I20260620 23:07:54.296489 6913 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:54.296576 6913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:54.296717 6913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:54.297716 6913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "ebfb647d261940e4a1a8a5720f2640ac"
format_stamp: "Formatted at 2026-06-20 23:07:54 on dist-test-slave-zpss"
server_key: "09e266bbf33117cd55718cb1d3056d25"
server_key_iv: "a0eb3785a2557cc18c26f6f968f20a19"
server_key_version: "encryptionkey@0"
I20260620 23:07:54.298045 6913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "ebfb647d261940e4a1a8a5720f2640ac"
format_stamp: "Formatted at 2026-06-20 23:07:54 on dist-test-slave-zpss"
server_key: "09e266bbf33117cd55718cb1d3056d25"
server_key_iv: "a0eb3785a2557cc18c26f6f968f20a19"
server_key_version: "encryptionkey@0"
I20260620 23:07:54.299290 6913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:54.299929 6928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:54.300087 6913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:54.300180 6913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "ebfb647d261940e4a1a8a5720f2640ac"
format_stamp: "Formatted at 2026-06-20 23:07:54 on dist-test-slave-zpss"
server_key: "09e266bbf33117cd55718cb1d3056d25"
server_key_iv: "a0eb3785a2557cc18c26f6f968f20a19"
server_key_version: "encryptionkey@0"
I20260620 23:07:54.300251 6913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:54.319013 6913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:54.321259 6913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:54.321434 6913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:54.325129 6913 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35799
I20260620 23:07:54.325161 6980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35799 every 8 connection(s)
I20260620 23:07:54.325505 6913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:54.326349 6981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:54.328545 6981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac: Bootstrap starting.
I20260620 23:07:54.329101 6981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:54.329450 6981 log.cc:824] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:54.330111 6981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac: No bootstrap required, opened a new log
I20260620 23:07:54.331648 6981 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfb647d261940e4a1a8a5720f2640ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } }
I20260620 23:07:54.331779 6981 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:54.331825 6981 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebfb647d261940e4a1a8a5720f2640ac, State: Initialized, Role: FOLLOWER
I20260620 23:07:54.331974 6981 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfb647d261940e4a1a8a5720f2640ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } }
I20260620 23:07:54.332058 6981 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:54.332110 6981 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:54.332160 6981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:54.332683 6981 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfb647d261940e4a1a8a5720f2640ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } }
I20260620 23:07:54.332790 6981 leader_election.cc:302] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebfb647d261940e4a1a8a5720f2640ac; no voters:
I20260620 23:07:54.332984 6981 leader_election.cc:288] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:54.333011 6984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:54.333190 6984 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [term 1 LEADER]: Becoming Leader. State: Replica: ebfb647d261940e4a1a8a5720f2640ac, State: Running, Role: LEADER
I20260620 23:07:54.333243 6981 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:54.333353 6984 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfb647d261940e4a1a8a5720f2640ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } }
I20260620 23:07:54.333727 6986 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebfb647d261940e4a1a8a5720f2640ac. Latest consensus state: current_term: 1 leader_uuid: "ebfb647d261940e4a1a8a5720f2640ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfb647d261940e4a1a8a5720f2640ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } } }
I20260620 23:07:54.333813 6986 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:54.333945 6985 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebfb647d261940e4a1a8a5720f2640ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfb647d261940e4a1a8a5720f2640ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } } }
I20260620 23:07:54.334051 6985 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:54.334050 6993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:54.334364 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 6913
I20260620 23:07:54.334492 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:54.334728 18161 external_mini_cluster.cc:1428] Setting key 23c84c91d91b3de77f5ba69bf92f470f
I20260620 23:07:54.334780 6993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:07:54 dist-test-slave-zpss krb5kdc[6900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:07:54.344838 7003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:54.335297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:46005 (local address 127.0.0.1:35799)
0620 23:07:54.335461 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:54.335470 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:54.335499 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:07:54.335611 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:54.335618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:54.335675 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:54.335807 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:54.336667 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:54.337610 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:54.338675 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:54.339083 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:54.339156 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:54.341656 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:54.341660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:54.341661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:54.341696 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:54.343303 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:54.343691 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:54.343694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:54.343695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:54.343790 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:54.344020 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:54.344021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:54.344021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:54.344254 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:54.344426 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:54.344511 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 23:07:54.344612 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260620 23:07:54.345047 6993 catalog_manager.cc:1347] Generated new cluster ID: 304f999cc9624767806eba9da0276199
I20260620 23:07:54.345091 6993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:54.345875 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:54.350013 6993 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:54.350625 6993 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:54.353775 6993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebfb647d261940e4a1a8a5720f2640ac: Generated new TSK 0
I20260620 23:07:54.353945 6993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:54.354494 7003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:54.347231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:33283 (local address 127.0.0.1:35799)
0620 23:07:54.347261 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:54.347263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:54.347369 (+ 106us) server_negotiation.cc:407] Connection header received
0620 23:07:54.347421 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:54.347422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:54.347430 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:54.347483 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:54.348085 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:54.348803 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:54.349850 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:54.350162 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:54.350225 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:54.351532 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:54.351534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:54.351534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:54.351541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:54.353083 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:54.353683 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:54.353684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:54.353684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:54.353742 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:54.353988 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:54.353989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:54.353989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:54.354119 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:54.354208 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:54.354394 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 23:07:54.354444 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:07:54.356428 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 6913
W20260620 23:07:54.360394 6898 connection.cc:535] client connection to 127.0.0.1:35799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35799): Connection reset by peer (error 104)
2026-06-20T23:07:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:54 dist-test-slave-zpss krb5kdc[7012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:54 dist-test-slave-zpss krb5kdc[7012](info): set up 2 sockets
Jun 20 23:07:54 dist-test-slave-zpss krb5kdc[7012](info): commencing operation
krb5kdc: starting...
W20260620 23:07:55.365391 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:55 dist-test-slave-zpss krb5kdc[7012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:55.472764 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:55.549682 7025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:55.549921 7025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:55.549957 7025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:55.551054 7025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:55.551119 7025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:55.551143 7025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:55.551170 7025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:55.551191 7025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:55.552665 7025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32935
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:55.552912 7025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:55.553115 7025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:55.557581 7031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:55.557646 7030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:55.557590 7033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:55.557830 7025 server_base.cc:1034] running on GCE node
I20260620 23:07:55.558043 7025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:55.558300 7025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:55.559434 7025 hybrid_clock.cc:648] HybridClock initialized: now 1781996875559398 us; error 55 us; skew 500 ppm
Jun 20 23:07:55 dist-test-slave-zpss krb5kdc[7012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:55.561704 7025 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:55.562109 7025 webserver.cc:460] Webserver started at http://127.0.0.1:34339/ using document root <none> and password file <none>
I20260620 23:07:55.562274 7025 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:55.562312 7025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:55.562391 7025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:55.563344 7025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "f41cc1ca7a534d0bba21f732baf50607"
format_stamp: "Formatted at 2026-06-20 23:07:55 on dist-test-slave-zpss"
server_key: "64036c7dc1cbcfe05118bb73e524f0f7"
server_key_iv: "8fcc2cd471df73ccd30764bbe7701a1e"
server_key_version: "encryptionkey@0"
I20260620 23:07:55.563630 7025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "f41cc1ca7a534d0bba21f732baf50607"
format_stamp: "Formatted at 2026-06-20 23:07:55 on dist-test-slave-zpss"
server_key: "64036c7dc1cbcfe05118bb73e524f0f7"
server_key_iv: "8fcc2cd471df73ccd30764bbe7701a1e"
server_key_version: "encryptionkey@0"
I20260620 23:07:55.564793 7025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:55.565464 7040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:55.565650 7025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:55.565737 7025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "f41cc1ca7a534d0bba21f732baf50607"
format_stamp: "Formatted at 2026-06-20 23:07:55 on dist-test-slave-zpss"
server_key: "64036c7dc1cbcfe05118bb73e524f0f7"
server_key_iv: "8fcc2cd471df73ccd30764bbe7701a1e"
server_key_version: "encryptionkey@0"
I20260620 23:07:55.565802 7025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:55.577692 7025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:55.579298 7025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:55.579439 7025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:55.583142 7025 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32935
I20260620 23:07:55.583227 7092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32935 every 8 connection(s)
I20260620 23:07:55.583491 7025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:55.584233 7093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:55.586251 7093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607: Bootstrap starting.
I20260620 23:07:55.586706 7093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:55.586925 7093 log.cc:824] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:55.587074 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7025
I20260620 23:07:55.587181 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:55.587376 18161 external_mini_cluster.cc:1428] Setting key 4e294657ebe1e5ca7b329159cf0edadd
I20260620 23:07:55.587628 7093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607: No bootstrap required, opened a new log
I20260620 23:07:55.590091 7093 raft_consensus.cc:348] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cc1ca7a534d0bba21f732baf50607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260620 23:07:55.590248 7093 raft_consensus.cc:374] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:55.590286 7093 raft_consensus.cc:729] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41cc1ca7a534d0bba21f732baf50607, State: Initialized, Role: FOLLOWER
I20260620 23:07:55.590392 7093 consensus_queue.cc:260] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cc1ca7a534d0bba21f732baf50607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260620 23:07:55.590458 7093 raft_consensus.cc:388] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:55.590485 7093 raft_consensus.cc:482] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:55.590528 7093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:55.591113 7093 raft_consensus.cc:504] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cc1ca7a534d0bba21f732baf50607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260620 23:07:55.591233 7093 leader_election.cc:302] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f41cc1ca7a534d0bba21f732baf50607; no voters:
I20260620 23:07:55.591430 7093 leader_election.cc:288] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:55.591450 7098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:55.591607 7098 raft_consensus.cc:686] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [term 1 LEADER]: Becoming Leader. State: Replica: f41cc1ca7a534d0bba21f732baf50607, State: Running, Role: LEADER
I20260620 23:07:55.591729 7098 consensus_queue.cc:237] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cc1ca7a534d0bba21f732baf50607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260620 23:07:55.591727 7093 sys_catalog.cc:564] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:55.592139 7100 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f41cc1ca7a534d0bba21f732baf50607. Latest consensus state: current_term: 1 leader_uuid: "f41cc1ca7a534d0bba21f732baf50607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cc1ca7a534d0bba21f732baf50607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } } }
I20260620 23:07:55.592124 7099 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f41cc1ca7a534d0bba21f732baf50607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41cc1ca7a534d0bba21f732baf50607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } } }
I20260620 23:07:55.592254 7100 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:55.592263 7099 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:55.593084 7102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:55.593255 7097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:55.587824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:55571 (local address 127.0.0.1:32935)
0620 23:07:55.587999 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:55.588007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:55.588029 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:07:55.588797 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:55.588817 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:55.588861 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:55.588981 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:55.589239 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:55.590093 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:55.591126 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:55.591685 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:55.591813 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:55.592476 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:55.592480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:55.592481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:55.592703 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:55.592757 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:55.592771 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:55.592850 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:55.593016 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 23:07:55.593056 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260620 23:07:55.593626 7102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:55.594225 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:55.595115 7102 catalog_manager.cc:1347] Generated new cluster ID: d6d1819844d84378afabc79af2885924
I20260620 23:07:55.595163 7102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:55.600749 7097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:55.595541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:45837 (local address 127.0.0.1:32935)
0620 23:07:55.595586 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:55.595587 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:55.597604 (+ 2017us) server_negotiation.cc:407] Connection header received
0620 23:07:55.597667 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:55.597668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:55.597678 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:55.597733 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:55.597847 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:55.598549 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:55.599566 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:55.599825 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:55.599880 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:55.600488 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:55.600489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:55.600490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:55.600493 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:55.600505 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:55.600508 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:55.600556 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:55.600673 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 23:07:55.600681 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:07:55.604712 7102 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:55.605374 7102 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:55.623735 7102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f41cc1ca7a534d0bba21f732baf50607: Generated new TSK 0
I20260620 23:07:55.623930 7102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:55.627199 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7025
W20260620 23:07:55.631078 7010 connection.cc:535] client connection to 127.0.0.1:32935 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32935): Connection reset by peer (error 104)
2026-06-20T23:07:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:55 dist-test-slave-zpss krb5kdc[7124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:55 dist-test-slave-zpss krb5kdc[7124](info): set up 2 sockets
Jun 20 23:07:55 dist-test-slave-zpss krb5kdc[7124](info): commencing operation
krb5kdc: starting...
W20260620 23:07:56.641188 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:56 dist-test-slave-zpss krb5kdc[7124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:07:56.748088 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:56.825410 7137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:56.825627 7137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:56.825664 7137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:56.826826 7137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:56.826892 7137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:56.826918 7137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:56.826947 7137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:56.826996 7137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:56.828409 7137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45251
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:56.828661 7137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:56.828871 7137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:56.833546 7142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:56.833587 7145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:56.833738 7143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:56.833715 7137 server_base.cc:1034] running on GCE node
I20260620 23:07:56.834008 7137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:56.834313 7137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:56.835484 7137 hybrid_clock.cc:648] HybridClock initialized: now 1781996876835471 us; error 26 us; skew 500 ppm
Jun 20 23:07:56 dist-test-slave-zpss krb5kdc[7124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:56.837814 7137 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:07:56.838199 7137 webserver.cc:460] Webserver started at http://127.0.0.1:45513/ using document root <none> and password file <none>
I20260620 23:07:56.838423 7137 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:56.838486 7137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:56.838639 7137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:56.839636 7137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "9f665e60a9974b29b6c9ed5ca0587d3f"
format_stamp: "Formatted at 2026-06-20 23:07:56 on dist-test-slave-zpss"
server_key: "357843cbb8817f3b118baf4a707f0466"
server_key_iv: "9102edc50ebfdd543a0d3e16a95bdb3b"
server_key_version: "encryptionkey@0"
I20260620 23:07:56.839977 7137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "9f665e60a9974b29b6c9ed5ca0587d3f"
format_stamp: "Formatted at 2026-06-20 23:07:56 on dist-test-slave-zpss"
server_key: "357843cbb8817f3b118baf4a707f0466"
server_key_iv: "9102edc50ebfdd543a0d3e16a95bdb3b"
server_key_version: "encryptionkey@0"
I20260620 23:07:56.841203 7137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:07:56.841948 7152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:56.842132 7137 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:07:56.842226 7137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "9f665e60a9974b29b6c9ed5ca0587d3f"
format_stamp: "Formatted at 2026-06-20 23:07:56 on dist-test-slave-zpss"
server_key: "357843cbb8817f3b118baf4a707f0466"
server_key_iv: "9102edc50ebfdd543a0d3e16a95bdb3b"
server_key_version: "encryptionkey@0"
I20260620 23:07:56.842309 7137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:56.845367 7137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:56.847020 7137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:56.847162 7137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:56.850870 7137 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45251
I20260620 23:07:56.850921 7204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45251 every 8 connection(s)
I20260620 23:07:56.851212 7137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:56.852025 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7137
I20260620 23:07:56.852039 7205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:56.852145 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:56.852315 18161 external_mini_cluster.cc:1428] Setting key 1f5269e192ab55113ba185605a552e4c
I20260620 23:07:56.854640 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:56.852800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:52649 (local address 127.0.0.1:45251)
0620 23:07:56.852968 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:56.852976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:56.853012 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:07:56.853480 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:56.853487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:56.853514 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:56.853621 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:56.854216 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:56.854219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:56.854219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:56.854222 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:56.854254 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:56.854263 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:56.854316 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:56.854398 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 23:07:56.854429 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260620 23:07:56.855263 7205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f: Bootstrap starting.
I20260620 23:07:56.855470 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:56.855967 7205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:56.856290 7205 log.cc:824] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:56.857000 7205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f: No bootstrap required, opened a new log
I20260620 23:07:56.858305 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:56.857060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:40001 (local address 127.0.0.1:45251)
0620 23:07:56.857117 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:56.857119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:56.857492 (+ 373us) server_negotiation.cc:407] Connection header received
0620 23:07:56.857564 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:56.857564 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:56.857568 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:56.857608 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:56.858087 (+ 479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:56.858088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:56.858089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:56.858092 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:56.858102 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:56.858105 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:56.858144 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:56.858208 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 23:07:56.858217 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:07:56.859308 7205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f665e60a9974b29b6c9ed5ca0587d3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45251 } }
I20260620 23:07:56.859458 7205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:56.859511 7205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f665e60a9974b29b6c9ed5ca0587d3f, State: Initialized, Role: FOLLOWER
I20260620 23:07:56.859634 7205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f665e60a9974b29b6c9ed5ca0587d3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45251 } }
I20260620 23:07:56.859737 7205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:56.859777 7205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:56.859826 7205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:56.860296 7205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f665e60a9974b29b6c9ed5ca0587d3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45251 } }
I20260620 23:07:56.860407 7205 leader_election.cc:302] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f665e60a9974b29b6c9ed5ca0587d3f; no voters:
I20260620 23:07:56.860602 7205 leader_election.cc:288] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:56.860646 7215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:56.860834 7215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [term 1 LEADER]: Becoming Leader. State: Replica: 9f665e60a9974b29b6c9ed5ca0587d3f, State: Running, Role: LEADER
I20260620 23:07:56.860843 7205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:56.860957 7215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f665e60a9974b29b6c9ed5ca0587d3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45251 } }
I20260620 23:07:56.861408 7216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f665e60a9974b29b6c9ed5ca0587d3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f665e60a9974b29b6c9ed5ca0587d3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45251 } } }
I20260620 23:07:56.861420 7217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f665e60a9974b29b6c9ed5ca0587d3f. Latest consensus state: current_term: 1 leader_uuid: "9f665e60a9974b29b6c9ed5ca0587d3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f665e60a9974b29b6c9ed5ca0587d3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45251 } } }
I20260620 23:07:56.861598 7217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:56.861583 7216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:56.862010 7225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:56.862454 7225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:56.864001 7225 catalog_manager.cc:1347] Generated new cluster ID: 0edab542aa254c1795b54c3dea9da283
I20260620 23:07:56.864051 7225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:56.871723 7225 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:56.871874 7225 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:56.875145 7225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f665e60a9974b29b6c9ed5ca0587d3f: Generated new TSK 0
I20260620 23:07:56.875288 7225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:56.881639 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7137
2026-06-20T23:07:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:56 dist-test-slave-zpss krb5kdc[7236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:56 dist-test-slave-zpss krb5kdc[7236](info): set up 2 sockets
Jun 20 23:07:56 dist-test-slave-zpss krb5kdc[7236](info): commencing operation
krb5kdc: starting...
W20260620 23:07:57.901477 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:57 dist-test-slave-zpss krb5kdc[7236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:07:58.016296 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42563
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:32811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:58.093456 7249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:58.093631 7249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:58.093652 7249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:58.094760 7249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:58.094800 7249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:58.094815 7249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:58.094861 7249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:58.094879 7249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:58.096298 7249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:32811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42563
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:58.096458 7249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:58.096670 7249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:58.101405 7255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:58.101553 7254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:58.101426 7257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:58.101579 7249 server_base.cc:1034] running on GCE node
I20260620 23:07:58.101894 7249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:58.102177 7249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:58.103341 7249 hybrid_clock.cc:648] HybridClock initialized: now 1781996878103332 us; error 39 us; skew 500 ppm
Jun 20 23:07:58 dist-test-slave-zpss krb5kdc[7236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996878, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:58.105580 7249 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:07:58.105912 7249 webserver.cc:460] Webserver started at http://127.17.188.126:44675/ using document root <none> and password file <none>
I20260620 23:07:58.106093 7249 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:58.106132 7249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:58.106212 7249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:58.107154 7249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "c9435bb874824bfa99ae4eb7ef885a28"
format_stamp: "Formatted at 2026-06-20 23:07:58 on dist-test-slave-zpss"
server_key: "2da529048754ef8d762be4de409e0e4f"
server_key_iv: "66485b46c412467f307b28dd97f3344c"
server_key_version: "encryptionkey@0"
I20260620 23:07:58.107462 7249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "c9435bb874824bfa99ae4eb7ef885a28"
format_stamp: "Formatted at 2026-06-20 23:07:58 on dist-test-slave-zpss"
server_key: "2da529048754ef8d762be4de409e0e4f"
server_key_iv: "66485b46c412467f307b28dd97f3344c"
server_key_version: "encryptionkey@0"
I20260620 23:07:58.108624 7249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:07:58.109194 7264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:58.109349 7249 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:58.109424 7249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "c9435bb874824bfa99ae4eb7ef885a28"
format_stamp: "Formatted at 2026-06-20 23:07:58 on dist-test-slave-zpss"
server_key: "2da529048754ef8d762be4de409e0e4f"
server_key_iv: "66485b46c412467f307b28dd97f3344c"
server_key_version: "encryptionkey@0"
I20260620 23:07:58.109470 7249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:58.147099 7249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:58.148737 7249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:58.148837 7249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:58.152493 7249 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:42563
I20260620 23:07:58.152550 7316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:42563 every 8 connection(s)
I20260620 23:07:58.152895 7249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:58.153750 7317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:58.155927 7317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28: Bootstrap starting.
I20260620 23:07:58.156525 7317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:58.156812 7317 log.cc:824] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:58.157506 7317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28: No bootstrap required, opened a new log
I20260620 23:07:58.159089 7317 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9435bb874824bfa99ae4eb7ef885a28" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42563 } }
I20260620 23:07:58.159243 7317 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:58.159289 7317 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9435bb874824bfa99ae4eb7ef885a28, State: Initialized, Role: FOLLOWER
I20260620 23:07:58.159471 7317 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9435bb874824bfa99ae4eb7ef885a28" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42563 } }
I20260620 23:07:58.159577 7317 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:58.159622 7317 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:58.159675 7317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:58.160280 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7249
I20260620 23:07:58.160413 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:58.160585 7317 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9435bb874824bfa99ae4eb7ef885a28" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42563 } }
I20260620 23:07:58.160619 18161 external_mini_cluster.cc:1428] Setting key 078f032ead7ec5a75c01cef46ab42465
I20260620 23:07:58.160698 7317 leader_election.cc:302] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9435bb874824bfa99ae4eb7ef885a28; no voters:
I20260620 23:07:58.160859 7317 leader_election.cc:288] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:58.160881 7320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:58.160996 7320 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [term 1 LEADER]: Becoming Leader. State: Replica: c9435bb874824bfa99ae4eb7ef885a28, State: Running, Role: LEADER
I20260620 23:07:58.161093 7320 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9435bb874824bfa99ae4eb7ef885a28" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42563 } }
I20260620 23:07:58.161121 7317 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:58.162328 7321 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9435bb874824bfa99ae4eb7ef885a28. Latest consensus state: current_term: 1 leader_uuid: "c9435bb874824bfa99ae4eb7ef885a28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9435bb874824bfa99ae4eb7ef885a28" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42563 } } }
I20260620 23:07:58.162483 7321 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:58.162837 7322 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9435bb874824bfa99ae4eb7ef885a28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9435bb874824bfa99ae4eb7ef885a28" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42563 } } }
I20260620 23:07:58.162914 7322 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28 [sys.catalog]: This master's current role is: LEADER
W20260620 23:07:58.163125 7336 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:07:58.163187 7336 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:07:58.163264 7338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:58.163439 7338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:58.165423 7338 catalog_manager.cc:1347] Generated new cluster ID: 6089e5987b8c43abb442ab6b9ba212c1
I20260620 23:07:58.165536 7338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:07:58 dist-test-slave-zpss krb5kdc[7236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:07:58.171064 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:58.161155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53143 (local address 127.17.188.126:42563)
0620 23:07:58.161298 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:58.161306 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:58.161534 (+ 228us) server_negotiation.cc:407] Connection header received
0620 23:07:58.161641 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:58.161646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:58.161685 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:58.161827 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:58.162633 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:58.163841 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:58.164940 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:58.165388 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:58.165507 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:58.167823 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:58.167826 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:58.167827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:58.167838 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:58.169564 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:58.169939 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:58.169942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:58.169943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:58.170043 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:58.170279 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:58.170280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:58.170280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:58.170511 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:58.170698 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:58.170792 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 23:07:58.170898 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260620 23:07:58.171964 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:58.183313 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:58.173630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38115 (local address 127.17.188.126:42563)
0620 23:07:58.173670 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:58.173672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:58.177491 (+ 3819us) server_negotiation.cc:407] Connection header received
0620 23:07:58.177557 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:58.177558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:58.177568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:58.177637 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:58.178138 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:58.178609 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:58.179305 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:58.179528 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:58.179602 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:58.180703 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:58.180704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:58.180705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:07:58.180712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:58.182141 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:58.182502 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:58.182503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:58.182503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:58.182571 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:07:58.182792 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:07:58.182793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:07:58.182793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:07:58.182902 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:58.183018 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:58.183139 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 23:07:58.183201 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:07:58.211331 7338 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:58.212095 7338 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:58.220295 7338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9435bb874824bfa99ae4eb7ef885a28: Generated new TSK 0
I20260620 23:07:58.220479 7338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:58.223587 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7249
W20260620 23:07:58.227756 7234 connection.cc:535] client connection to 127.17.188.126:42563 recv error: Network error: failed to read from TLS socket (remote: 127.17.188.126:42563): Connection reset by peer (error 104)
2026-06-20T23:07:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:58 dist-test-slave-zpss krb5kdc[7348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:58 dist-test-slave-zpss krb5kdc[7348](info): set up 2 sockets
Jun 20 23:07:58 dist-test-slave-zpss krb5kdc[7348](info): commencing operation
krb5kdc: starting...
W20260620 23:07:59.245404 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:07:59 dist-test-slave-zpss krb5kdc[7348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:07:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:07:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:07:59.353080 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:38137
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:38137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:07:59.430788 7361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:07:59.430972 7361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:07:59.430995 7361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:07:59.432124 7361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:07:59.432164 7361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:07:59.432179 7361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:07:59.432191 7361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:07:59.432245 7361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:07:59.433782 7361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:38137
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:38137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:07:59.434012 7361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:07:59.434218 7361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:07:59.438838 7366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:59.438851 7369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:07:59.438838 7367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:07:59.439095 7361 server_base.cc:1034] running on GCE node
I20260620 23:07:59.439368 7361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:07:59.439674 7361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:07:59.440819 7361 hybrid_clock.cc:648] HybridClock initialized: now 1781996879440794 us; error 41 us; skew 500 ppm
Jun 20 23:07:59 dist-test-slave-zpss krb5kdc[7348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996879, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:07:59.446430 7361 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:07:59.446892 7361 webserver.cc:460] Webserver started at http://127.17.188.126:35891/ using document root <none> and password file <none>
I20260620 23:07:59.447115 7361 fs_manager.cc:359] Metadata directory not provided
I20260620 23:07:59.447180 7361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:07:59.447320 7361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:07:59.448343 7361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "5a73030e095546d78dab58663e258d14"
format_stamp: "Formatted at 2026-06-20 23:07:59 on dist-test-slave-zpss"
server_key: "aeb236ecbab121d741a79b8f80586b7b"
server_key_iv: "663041c9bf82afd34e3c8cd2fdf18e21"
server_key_version: "encryptionkey@0"
I20260620 23:07:59.448693 7361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "5a73030e095546d78dab58663e258d14"
format_stamp: "Formatted at 2026-06-20 23:07:59 on dist-test-slave-zpss"
server_key: "aeb236ecbab121d741a79b8f80586b7b"
server_key_iv: "663041c9bf82afd34e3c8cd2fdf18e21"
server_key_version: "encryptionkey@0"
I20260620 23:07:59.449996 7361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:07:59.450687 7376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:07:59.450850 7361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:07:59.450954 7361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "5a73030e095546d78dab58663e258d14"
format_stamp: "Formatted at 2026-06-20 23:07:59 on dist-test-slave-zpss"
server_key: "aeb236ecbab121d741a79b8f80586b7b"
server_key_iv: "663041c9bf82afd34e3c8cd2fdf18e21"
server_key_version: "encryptionkey@0"
I20260620 23:07:59.451025 7361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:07:59.469882 7361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:07:59.471838 7361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:07:59.472013 7361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:07:59.475828 7361 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:38137
I20260620 23:07:59.475862 7428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:38137 every 8 connection(s)
I20260620 23:07:59.476203 7361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:07:59.477046 7429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:07:59.477074 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7361
I20260620 23:07:59.477205 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:07:59.477425 18161 external_mini_cluster.cc:1428] Setting key 84981cc6909b0bfd6b8db1a5aa724151
I20260620 23:07:59.480077 7429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14: Bootstrap starting.
I20260620 23:07:59.480731 7429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14: Neither blocks nor log segments found. Creating new log.
I20260620 23:07:59.481027 7429 log.cc:824] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14: Log is configured to *not* fsync() on all Append() calls
I20260620 23:07:59.481773 7429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14: No bootstrap required, opened a new log
I20260620 23:07:59.482787 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:59.477932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40917 (local address 127.17.188.126:38137)
0620 23:07:59.478088 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:59.478095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:59.478118 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:07:59.478536 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:59.478542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:59.478583 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:59.478688 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:59.478924 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:59.479768 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:59.480877 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:59.481303 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:59.481476 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:59.482185 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:59.482190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:59.482190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:59.482344 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:59.482371 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:59.482381 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:59.482478 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:59.482600 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 23:07:59.482628 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260620 23:07:59.483479 7429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a73030e095546d78dab58663e258d14" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38137 } }
I20260620 23:07:59.483675 7429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:07:59.483718 7429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a73030e095546d78dab58663e258d14, State: Initialized, Role: FOLLOWER
I20260620 23:07:59.483826 7429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a73030e095546d78dab58663e258d14" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38137 } }
I20260620 23:07:59.483906 7429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:07:59.483940 7429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:07:59.483984 7429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:07:59.484117 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:07:59.484534 7429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a73030e095546d78dab58663e258d14" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38137 } }
I20260620 23:07:59.484660 7429 leader_election.cc:302] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a73030e095546d78dab58663e258d14; no voters:
I20260620 23:07:59.484844 7429 leader_election.cc:288] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:07:59.484874 7434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:07:59.484977 7434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [term 1 LEADER]: Becoming Leader. State: Replica: 5a73030e095546d78dab58663e258d14, State: Running, Role: LEADER
I20260620 23:07:59.485086 7429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:07:59.485105 7434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a73030e095546d78dab58663e258d14" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38137 } }
I20260620 23:07:59.485558 7434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a73030e095546d78dab58663e258d14. Latest consensus state: current_term: 1 leader_uuid: "5a73030e095546d78dab58663e258d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a73030e095546d78dab58663e258d14" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38137 } } }
I20260620 23:07:59.485666 7434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:59.485785 7436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a73030e095546d78dab58663e258d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a73030e095546d78dab58663e258d14" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 38137 } } }
I20260620 23:07:59.485872 7436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14 [sys.catalog]: This master's current role is: LEADER
I20260620 23:07:59.485888 7444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:07:59.486351 7444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:07:59.487965 7444 catalog_manager.cc:1347] Generated new cluster ID: 5e1962145cdd42b6bf55bfc43a865f70
I20260620 23:07:59.488018 7444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:07:59.490160 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:07:59.485568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54707 (local address 127.17.188.126:38137)
0620 23:07:59.485651 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 23:07:59.485654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:07:59.486834 (+ 1180us) server_negotiation.cc:407] Connection header received
0620 23:07:59.486944 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:07:59.486945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:07:59.486954 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:07:59.486984 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:07:59.487186 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:59.487776 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:59.488866 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:07:59.489164 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:07:59.489196 (+ 32us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:07:59.489900 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:07:59.489901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:07:59.489901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:07:59.489905 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:07:59.489917 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:07:59.489920 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:07:59.489958 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 23:07:59.490069 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 23:07:59.490078 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 23:07:59.514808 7444 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:07:59.515367 7444 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:07:59.517650 7444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a73030e095546d78dab58663e258d14: Generated new TSK 0
I20260620 23:07:59.517827 7444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:07:59.520774 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7361
W20260620 23:07:59.524811 7346 connection.cc:535] client connection to 127.17.188.126:38137 recv error: Network error: failed to read from TLS socket (remote: 127.17.188.126:38137): Connection reset by peer (error 104)
2026-06-20T23:07:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:07:59 dist-test-slave-zpss krb5kdc[7460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:07:59 dist-test-slave-zpss krb5kdc[7460](info): set up 2 sockets
Jun 20 23:07:59 dist-test-slave-zpss krb5kdc[7460](info): commencing operation
krb5kdc: starting...
W20260620 23:08:00.529578 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:08:00 dist-test-slave-zpss krb5kdc[7460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:08:00.641561 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:37715
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:42147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:37715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:00.718695 7473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:00.718928 7473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:00.718974 7473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:00.720090 7473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:00.720156 7473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:00.720180 7473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:00.720207 7473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:00.720235 7473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:00.721717 7473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:37715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:37715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:00.721988 7473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:00.722201 7473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:00.726840 7478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:00.726845 7479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:00.726847 7481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:00.727109 7473 server_base.cc:1034] running on GCE node
I20260620 23:08:00.727326 7473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:00.727562 7473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:00.728744 7473 hybrid_clock.cc:648] HybridClock initialized: now 1781996880728731 us; error 29 us; skew 500 ppm
Jun 20 23:08:00 dist-test-slave-zpss krb5kdc[7460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996880, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:00.731235 7473 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:08:00.731686 7473 webserver.cc:460] Webserver started at http://127.17.188.126:40133/ using document root <none> and password file <none>
I20260620 23:08:00.731913 7473 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:00.731963 7473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:00.732136 7473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:00.733132 7473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d"
format_stamp: "Formatted at 2026-06-20 23:08:00 on dist-test-slave-zpss"
server_key: "8c6fd947f6acaaaa4cd19d097ef9c524"
server_key_iv: "172e87a13a4e79c1063b10883ba25480"
server_key_version: "encryptionkey@0"
I20260620 23:08:00.733520 7473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d"
format_stamp: "Formatted at 2026-06-20 23:08:00 on dist-test-slave-zpss"
server_key: "8c6fd947f6acaaaa4cd19d097ef9c524"
server_key_iv: "172e87a13a4e79c1063b10883ba25480"
server_key_version: "encryptionkey@0"
I20260620 23:08:00.734783 7473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:00.735478 7488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:00.735659 7473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:00.735747 7473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d"
format_stamp: "Formatted at 2026-06-20 23:08:00 on dist-test-slave-zpss"
server_key: "8c6fd947f6acaaaa4cd19d097ef9c524"
server_key_iv: "172e87a13a4e79c1063b10883ba25480"
server_key_version: "encryptionkey@0"
I20260620 23:08:00.735821 7473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:00.739076 7473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:00.741358 7473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:00.741520 7473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:00.745292 7473 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:37715
I20260620 23:08:00.745373 7540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:37715 every 8 connection(s)
I20260620 23:08:00.745747 7473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:00.746706 7541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:00.748764 7541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d: Bootstrap starting.
I20260620 23:08:00.749282 7541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:00.749574 7541 log.cc:824] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:00.750191 7541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d: No bootstrap required, opened a new log
I20260620 23:08:00.751693 7541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 37715 } }
I20260620 23:08:00.751824 7541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:00.751845 7541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36aaa50fdc5d4a39ba509fdbcf33df5d, State: Initialized, Role: FOLLOWER
I20260620 23:08:00.751950 7541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 37715 } }
I20260620 23:08:00.752002 7541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:00.752022 7541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:00.752048 7541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:00.752776 7541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 37715 } }
I20260620 23:08:00.752857 7541 leader_election.cc:302] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36aaa50fdc5d4a39ba509fdbcf33df5d; no voters:
I20260620 23:08:00.752991 7541 leader_election.cc:288] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:00.753067 7544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:00.753223 7544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [term 1 LEADER]: Becoming Leader. State: Replica: 36aaa50fdc5d4a39ba509fdbcf33df5d, State: Running, Role: LEADER
I20260620 23:08:00.753258 7541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:00.753309 7544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 37715 } }
I20260620 23:08:00.753719 7545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 37715 } } }
I20260620 23:08:00.753746 7546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36aaa50fdc5d4a39ba509fdbcf33df5d. Latest consensus state: current_term: 1 leader_uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36aaa50fdc5d4a39ba509fdbcf33df5d" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 37715 } } }
I20260620 23:08:00.753859 7545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:00.754251 7546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:00.754328 7553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:00.754899 7553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:00.755905 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7473
I20260620 23:08:00.756023 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:08:00.756232 18161 external_mini_cluster.cc:1428] Setting key a645f36ddc86808066fbb72354d3ef0e
I20260620 23:08:00.756793 7553 catalog_manager.cc:1347] Generated new cluster ID: c289b8da83eb494ca2f91dbe6126e3f1
I20260620 23:08:00.756841 7553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:00.759182 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:00.756683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33847 (local address 127.17.188.126:37715)
0620 23:08:00.757690 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:00.757698 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:00.757728 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:08:00.757856 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:00.757863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:00.757891 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:00.758024 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:00.758727 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:00.758731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:00.758731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:00.758736 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:00.758777 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:00.758791 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:00.758878 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:00.758955 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 23:08:00.758996 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":967,"thread_start_us":77,"threads_started":1}
I20260620 23:08:00.760109 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:00.762709 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:00.761735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52723 (local address 127.17.188.126:37715)
0620 23:08:00.761783 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:00.761784 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:00.761792 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:00.761860 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:00.761860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:00.761864 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:00.761907 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:00.762504 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:00.762506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:00.762506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:00.762510 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:00.762523 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:00.762527 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:00.762574 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:00.762643 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:08:00.762655 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:08:00.778133 7553 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:00.778293 7553 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:00.784327 7553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36aaa50fdc5d4a39ba509fdbcf33df5d: Generated new TSK 0
I20260620 23:08:00.784500 7553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:08:00.789790 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7473
2026-06-20T23:08:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1269 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:00 dist-test-slave-zpss krb5kdc[7572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:00 dist-test-slave-zpss krb5kdc[7572](info): set up 2 sockets
Jun 20 23:08:00 dist-test-slave-zpss krb5kdc[7572](info): commencing operation
krb5kdc: starting...
W20260620 23:08:01.813359 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:08:01 dist-test-slave-zpss krb5kdc[7572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:08:01.923280 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:41395
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:41395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:02.000927 7585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:02.001107 7585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:02.001129 7585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:02.002296 7585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:02.002338 7585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:02.002352 7585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:02.002386 7585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:02.002415 7585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:02.003840 7585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:41395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:41395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:02.004043 7585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:02.004223 7585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:02.008984 7591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:02.008998 7593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:02.008971 7590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:02.009019 7585 server_base.cc:1034] running on GCE node
I20260620 23:08:02.009485 7585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:02.009789 7585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:02.010931 7585 hybrid_clock.cc:648] HybridClock initialized: now 1781996882010922 us; error 29 us; skew 500 ppm
Jun 20 23:08:02 dist-test-slave-zpss krb5kdc[7572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996882, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:02.013254 7585 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:08:02.013691 7585 webserver.cc:460] Webserver started at http://127.17.188.126:44691/ using document root <none> and password file <none>
I20260620 23:08:02.013859 7585 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:02.013904 7585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:02.013984 7585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:02.014891 7585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "7cc8bceb384544b49922586ff8f5f2d7"
format_stamp: "Formatted at 2026-06-20 23:08:02 on dist-test-slave-zpss"
server_key: "34cc263b3d03be711307af6cc161838a"
server_key_iv: "361313019fc5806981392532ad486b64"
server_key_version: "encryptionkey@0"
I20260620 23:08:02.015178 7585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "7cc8bceb384544b49922586ff8f5f2d7"
format_stamp: "Formatted at 2026-06-20 23:08:02 on dist-test-slave-zpss"
server_key: "34cc263b3d03be711307af6cc161838a"
server_key_iv: "361313019fc5806981392532ad486b64"
server_key_version: "encryptionkey@0"
I20260620 23:08:02.016311 7585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:02.016942 7600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:02.017117 7585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:02.017208 7585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "7cc8bceb384544b49922586ff8f5f2d7"
format_stamp: "Formatted at 2026-06-20 23:08:02 on dist-test-slave-zpss"
server_key: "34cc263b3d03be711307af6cc161838a"
server_key_iv: "361313019fc5806981392532ad486b64"
server_key_version: "encryptionkey@0"
I20260620 23:08:02.017302 7585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:02.038127 7585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:02.039914 7585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:02.040067 7585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:02.044091 7585 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:41395
I20260620 23:08:02.044124 7652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:41395 every 8 connection(s)
I20260620 23:08:02.044481 7585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:02.045281 7653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:02.047523 7653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7: Bootstrap starting.
I20260620 23:08:02.047542 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7585
I20260620 23:08:02.047667 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:08:02.047855 18161 external_mini_cluster.cc:1428] Setting key 1ee60c111729945b392d8546eb4ba9a0
I20260620 23:08:02.048218 7653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:02.048681 7653 log.cc:824] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:02.049464 7653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7: No bootstrap required, opened a new log
I20260620 23:08:02.051671 7653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8bceb384544b49922586ff8f5f2d7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41395 } }
I20260620 23:08:02.051810 7653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:02.051834 7653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc8bceb384544b49922586ff8f5f2d7, State: Initialized, Role: FOLLOWER
I20260620 23:08:02.051975 7653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8bceb384544b49922586ff8f5f2d7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41395 } }
I20260620 23:08:02.052045 7653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:02.052067 7653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:02.052134 7653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:02.052747 7653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8bceb384544b49922586ff8f5f2d7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41395 } }
I20260620 23:08:02.052847 7653 leader_election.cc:302] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cc8bceb384544b49922586ff8f5f2d7; no voters:
I20260620 23:08:02.052997 7653 leader_election.cc:288] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:02.053064 7658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:02.053236 7658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc8bceb384544b49922586ff8f5f2d7, State: Running, Role: LEADER
I20260620 23:08:02.053239 7653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:02.053361 7658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8bceb384544b49922586ff8f5f2d7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41395 } }
I20260620 23:08:02.053675 7658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cc8bceb384544b49922586ff8f5f2d7. Latest consensus state: current_term: 1 leader_uuid: "7cc8bceb384544b49922586ff8f5f2d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8bceb384544b49922586ff8f5f2d7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41395 } } }
I20260620 23:08:02.053745 7658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:02.053910 7659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cc8bceb384544b49922586ff8f5f2d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8bceb384544b49922586ff8f5f2d7" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 41395 } } }
I20260620 23:08:02.053977 7659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:02.054095 7663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:02.054590 7663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:08:02 dist-test-slave-zpss krb5kdc[7572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:08:02.056206 7663 catalog_manager.cc:1347] Generated new cluster ID: 11629672f30748acadd85446b9462197
I20260620 23:08:02.056257 7663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:02.060197 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:02.048358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49049 (local address 127.17.188.126:41395)
0620 23:08:02.048619 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:02.048628 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:02.048662 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:08:02.049264 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:02.049273 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:02.049331 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:02.049503 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:02.050085 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:02.051511 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:02.052473 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:02.053105 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:02.053205 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:02.056084 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:02.056087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:02.056088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:02.056098 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:02.058045 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:02.058445 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:02.058448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:02.058448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:02.058559 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:02.058758 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:02.058758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:02.058759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:02.059131 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:02.059314 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:02.059855 (+ 541us) server_negotiation.cc:299] Negotiation successful
0620 23:08:02.059991 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":105,"threads_started":1}
I20260620 23:08:02.062882 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:02.063560 7663 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:02.064224 7663 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:02.070447 7663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cc8bceb384544b49922586ff8f5f2d7: Generated new TSK 0
I20260620 23:08:02.070627 7663 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:08:02.071246 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:02.064333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55721 (local address 127.17.188.126:41395)
0620 23:08:02.064394 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:02.064396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:02.064539 (+ 143us) server_negotiation.cc:407] Connection header received
0620 23:08:02.064564 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:02.064564 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:02.064573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:02.064639 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:02.065193 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:02.065927 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:02.066899 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:02.067218 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:02.067290 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:02.068469 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:02.068471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:02.068471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:02.068478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:02.069991 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:02.070474 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:02.070475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:02.070476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:02.070531 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:02.070746 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:02.070746 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:02.070747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:02.070857 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:02.070926 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:02.071129 (+ 203us) server_negotiation.cc:299] Negotiation successful
0620 23:08:02.071193 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:08:02.073230 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7585
2026-06-20T23:08:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:02 dist-test-slave-zpss krb5kdc[7684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:02 dist-test-slave-zpss krb5kdc[7684](info): set up 2 sockets
Jun 20 23:08:02 dist-test-slave-zpss krb5kdc[7684](info): commencing operation
krb5kdc: starting...
W20260620 23:08:03.097576 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:08:03 dist-test-slave-zpss krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:08:03.211587 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35359
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:35359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:03.290679 7697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:03.290910 7697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:03.290935 7697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:03.292065 7697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:03.292112 7697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:03.292129 7697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:03.292191 7697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:03.292222 7697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:03.293711 7697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:35359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:35359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:03.293887 7697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:03.294124 7697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:03.298951 7702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:03.298923 7705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:03.298977 7697 server_base.cc:1034] running on GCE node
W20260620 23:08:03.298926 7703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:03.299463 7697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:03.299755 7697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:03.300935 7697 hybrid_clock.cc:648] HybridClock initialized: now 1781996883300930 us; error 38 us; skew 500 ppm
Jun 20 23:08:03 dist-test-slave-zpss krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996883, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:03.303372 7697 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:08:03.303736 7697 webserver.cc:460] Webserver started at http://127.17.188.126:36425/ using document root <none> and password file <none>
I20260620 23:08:03.303903 7697 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:03.303942 7697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:03.304025 7697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:03.304986 7697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "0be041db8ca841949e75bbf178d80e2e"
format_stamp: "Formatted at 2026-06-20 23:08:03 on dist-test-slave-zpss"
server_key: "ef67e359ba19a96596c41bece154db1a"
server_key_iv: "6d7a0923b6a35ef9aa749de3382319c6"
server_key_version: "encryptionkey@0"
I20260620 23:08:03.305271 7697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "0be041db8ca841949e75bbf178d80e2e"
format_stamp: "Formatted at 2026-06-20 23:08:03 on dist-test-slave-zpss"
server_key: "ef67e359ba19a96596c41bece154db1a"
server_key_iv: "6d7a0923b6a35ef9aa749de3382319c6"
server_key_version: "encryptionkey@0"
I20260620 23:08:03.306543 7697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:03.307161 7712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:03.307309 7697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:03.307391 7697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "0be041db8ca841949e75bbf178d80e2e"
format_stamp: "Formatted at 2026-06-20 23:08:03 on dist-test-slave-zpss"
server_key: "ef67e359ba19a96596c41bece154db1a"
server_key_iv: "6d7a0923b6a35ef9aa749de3382319c6"
server_key_version: "encryptionkey@0"
I20260620 23:08:03.307447 7697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:03.317143 7697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:03.318810 7697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:03.318909 7697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:03.322511 7697 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:35359
I20260620 23:08:03.322561 7764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:35359 every 8 connection(s)
I20260620 23:08:03.322870 7697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:03.323673 7765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:03.325836 7765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e: Bootstrap starting.
I20260620 23:08:03.326159 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7697
I20260620 23:08:03.326292 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:08:03.326500 18161 external_mini_cluster.cc:1428] Setting key c54dc9739033834fbcee31c6cb7ef130
I20260620 23:08:03.326567 7765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:03.326821 7765 log.cc:824] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:03.327589 7765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e: No bootstrap required, opened a new log
I20260620 23:08:03.329733 7765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be041db8ca841949e75bbf178d80e2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35359 } }
I20260620 23:08:03.329888 7765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:03.329921 7765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be041db8ca841949e75bbf178d80e2e, State: Initialized, Role: FOLLOWER
I20260620 23:08:03.330034 7765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be041db8ca841949e75bbf178d80e2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35359 } }
I20260620 23:08:03.330103 7765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:03.330139 7765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:03.330193 7765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:03.330755 7765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be041db8ca841949e75bbf178d80e2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35359 } }
I20260620 23:08:03.330866 7765 leader_election.cc:302] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0be041db8ca841949e75bbf178d80e2e; no voters:
I20260620 23:08:03.331029 7765 leader_election.cc:288] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:03.331073 7770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:03.331245 7770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [term 1 LEADER]: Becoming Leader. State: Replica: 0be041db8ca841949e75bbf178d80e2e, State: Running, Role: LEADER
I20260620 23:08:03.331317 7765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:03.331841 7770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be041db8ca841949e75bbf178d80e2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35359 } }
I20260620 23:08:03.332345 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:03.326980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37411 (local address 127.17.188.126:35359)
0620 23:08:03.327171 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:03.327180 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:03.327257 (+ 77us) server_negotiation.cc:407] Connection header received
0620 23:08:03.327757 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:03.327764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:03.327823 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:03.327966 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:03.328212 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:03.329042 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:03.330197 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:03.330642 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:03.330718 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:03.331437 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:03.331441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:03.331442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:03.331636 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:03.331672 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:03.331686 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:03.331745 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:03.332066 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 23:08:03.332143 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260620 23:08:03.332460 7771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0be041db8ca841949e75bbf178d80e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be041db8ca841949e75bbf178d80e2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35359 } } }
I20260620 23:08:03.332535 7771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:03.332892 7772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0be041db8ca841949e75bbf178d80e2e. Latest consensus state: current_term: 1 leader_uuid: "0be041db8ca841949e75bbf178d80e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be041db8ca841949e75bbf178d80e2e" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 35359 } } }
I20260620 23:08:03.332978 7772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e [sys.catalog]: This master's current role is: LEADER
W20260620 23:08:03.333288 7784 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:03.333427 7784 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:03.333495 7785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:03.333559 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:03.333701 7785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:03.335573 7785 catalog_manager.cc:1347] Generated new cluster ID: dc42bcacfec94da7bd188326ff2a8ad4
I20260620 23:08:03.335626 7785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:03.338315 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:03.334940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56665 (local address 127.17.188.126:35359)
0620 23:08:03.334969 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:03.334970 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:03.335167 (+ 197us) server_negotiation.cc:407] Connection header received
0620 23:08:03.335204 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:03.335204 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:03.335213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:03.335276 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:03.335435 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:03.336150 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:03.337087 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:03.337380 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:03.337414 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:03.338111 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:03.338112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:03.338113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:03.338116 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:03.338127 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:03.338130 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:03.338153 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:03.338234 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 23:08:03.338252 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260620 23:08:03.346529 7785 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:03.347222 7785 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:03.353494 7785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0be041db8ca841949e75bbf178d80e2e: Generated new TSK 0
I20260620 23:08:03.353641 7785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:08:03.355867 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7697
2026-06-20T23:08:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:03 dist-test-slave-zpss krb5kdc[7796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:03 dist-test-slave-zpss krb5kdc[7796](info): set up 2 sockets
Jun 20 23:08:03 dist-test-slave-zpss krb5kdc[7796](info): commencing operation
krb5kdc: starting...
W20260620 23:08:04.365437 18161 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:08:04 dist-test-slave-zpss krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:08:04.475723 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:43337
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:43337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:04.552448 7809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:04.552680 7809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:04.552717 7809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:04.553844 7809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:04.553923 7809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:04.553952 7809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:04.553982 7809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:04.554001 7809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:04.555388 7809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:43337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:43337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:04.555644 7809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:04.555861 7809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:04.560458 7814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:04.560468 7815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:04.560606 7809 server_base.cc:1034] running on GCE node
W20260620 23:08:04.560468 7817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:04.560935 7809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:04.561211 7809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:04.562350 7809 hybrid_clock.cc:648] HybridClock initialized: now 1781996884562325 us; error 43 us; skew 500 ppm
Jun 20 23:08:04 dist-test-slave-zpss krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996884, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:04.567113 7809 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:08:04.567545 7809 webserver.cc:460] Webserver started at http://127.17.188.126:45263/ using document root <none> and password file <none>
I20260620 23:08:04.567708 7809 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:04.567746 7809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:04.567827 7809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:04.568738 7809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "a6fc2f09b0ef49d08b1982e263416870"
format_stamp: "Formatted at 2026-06-20 23:08:04 on dist-test-slave-zpss"
server_key: "64b5093059fd9ed0e29d11ce03cbc963"
server_key_iv: "16f647c6c404ecdd104d83f795e5d4f0"
server_key_version: "encryptionkey@0"
I20260620 23:08:04.569016 7809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "a6fc2f09b0ef49d08b1982e263416870"
format_stamp: "Formatted at 2026-06-20 23:08:04 on dist-test-slave-zpss"
server_key: "64b5093059fd9ed0e29d11ce03cbc963"
server_key_iv: "16f647c6c404ecdd104d83f795e5d4f0"
server_key_version: "encryptionkey@0"
I20260620 23:08:04.570313 7809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:04.570955 7824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:04.571115 7809 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:04.571210 7809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "a6fc2f09b0ef49d08b1982e263416870"
format_stamp: "Formatted at 2026-06-20 23:08:04 on dist-test-slave-zpss"
server_key: "64b5093059fd9ed0e29d11ce03cbc963"
server_key_iv: "16f647c6c404ecdd104d83f795e5d4f0"
server_key_version: "encryptionkey@0"
I20260620 23:08:04.571285 7809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:04.574435 7809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:04.576098 7809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:04.576249 7809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:04.580065 7809 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:43337
I20260620 23:08:04.580096 7876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:43337 every 8 connection(s)
I20260620 23:08:04.580382 7809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:04.581240 7877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:04.583294 7877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870: Bootstrap starting.
I20260620 23:08:04.583827 7877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:04.584110 7877 log.cc:824] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:04.584733 7877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870: No bootstrap required, opened a new log
I20260620 23:08:04.586314 7877 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6fc2f09b0ef49d08b1982e263416870" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 43337 } }
I20260620 23:08:04.586443 7877 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:04.586467 7877 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fc2f09b0ef49d08b1982e263416870, State: Initialized, Role: FOLLOWER
I20260620 23:08:04.586649 7877 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6fc2f09b0ef49d08b1982e263416870" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 43337 } }
I20260620 23:08:04.586727 7877 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:04.586748 7877 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:04.586812 7877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:04.587603 7877 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6fc2f09b0ef49d08b1982e263416870" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 43337 } }
I20260620 23:08:04.587713 7877 leader_election.cc:302] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6fc2f09b0ef49d08b1982e263416870; no voters:
I20260620 23:08:04.587908 7877 leader_election.cc:288] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:04.587982 7880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:04.588155 7877 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:04.588155 7880 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [term 1 LEADER]: Becoming Leader. State: Replica: a6fc2f09b0ef49d08b1982e263416870, State: Running, Role: LEADER
I20260620 23:08:04.588296 7880 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6fc2f09b0ef49d08b1982e263416870" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 43337 } }
I20260620 23:08:04.588642 7882 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6fc2f09b0ef49d08b1982e263416870. Latest consensus state: current_term: 1 leader_uuid: "a6fc2f09b0ef49d08b1982e263416870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6fc2f09b0ef49d08b1982e263416870" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 43337 } } }
I20260620 23:08:04.588723 7882 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:04.588958 7889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:04.588912 7881 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6fc2f09b0ef49d08b1982e263416870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6fc2f09b0ef49d08b1982e263416870" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 43337 } } }
I20260620 23:08:04.589228 7881 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:04.589504 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7809
I20260620 23:08:04.589547 7889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:04.589615 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:08:04.589802 18161 external_mini_cluster.cc:1428] Setting key 4e9f231a73d7b4fac8b73be429e1e349
I20260620 23:08:04.594187 7889 catalog_manager.cc:1347] Generated new cluster ID: 22bc52dcafd04db893bef1ae559af2be
I20260620 23:08:04.594250 7889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:04.595049 7899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:04.590355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49701 (local address 127.17.188.126:43337)
0620 23:08:04.590512 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:04.590520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:04.593537 (+ 3017us) server_negotiation.cc:407] Connection header received
0620 23:08:04.593686 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:04.593693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:04.593734 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:04.593873 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:04.594494 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:04.594497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:04.594497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:04.594501 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:04.594534 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:04.594546 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:04.594589 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:04.594719 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 23:08:04.594753 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260620 23:08:04.595929 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:04.608054 7889 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:04.608213 7889 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:04.608814 7899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:04.597301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55237 (local address 127.17.188.126:43337)
0620 23:08:04.597353 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:04.597354 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:04.607841 (+ 10487us) server_negotiation.cc:407] Connection header received
0620 23:08:04.607950 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:04.607951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:04.607957 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:04.608012 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:04.608603 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:04.608604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:04.608605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:04.608609 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:04.608623 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:04.608627 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:04.608683 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:04.608745 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 23:08:04.608757 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:08:04.611423 7889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6fc2f09b0ef49d08b1982e263416870: Generated new TSK 0
I20260620 23:08:04.611594 7889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:08:04.616379 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7809
2026-06-20T23:08:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1260 ms)
[----------] 18 tests from AuthTokenIssuingTest (22978 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:04 dist-test-slave-zpss krb5kdc[7908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:04 dist-test-slave-zpss krb5kdc[7908](info): set up 2 sockets
Jun 20 23:08:04 dist-test-slave-zpss krb5kdc[7908](info): commencing operation
krb5kdc: starting...
W20260620 23:08:05.641283 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:08:05 dist-test-slave-zpss krb5kdc[7908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:08:05.751312 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42669
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42669,127.17.188.125:35307,127.17.188.124:43367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:05.828301 7921 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:08:05.828524 7921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:05.828553 7921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:05.828591 7921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:05.829768 7921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:05.829828 7921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:05.829860 7921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:05.829891 7921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:05.829911 7921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:05.831363 7921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42669,127.17.188.125:35307,127.17.188.124:43367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:42669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:05.831631 7921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:05.831851 7921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:05.836676 7927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:05.836779 7921 server_base.cc:1034] running on GCE node
W20260620 23:08:05.836747 7929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:05.836675 7926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:05.837126 7921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:05.837436 7921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:05.838591 7921 hybrid_clock.cc:648] HybridClock initialized: now 1781996885838583 us; error 28 us; skew 500 ppm
Jun 20 23:08:05 dist-test-slave-zpss krb5kdc[7908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:05.840912 7921 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:08:05.841251 7921 webserver.cc:460] Webserver started at http://127.17.188.126:35525/ using document root <none> and password file <none>
I20260620 23:08:05.841483 7921 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:05.841527 7921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:05.841609 7921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:05.842556 7921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "b534e26442f840ee835cffa6e5befae3"
format_stamp: "Formatted at 2026-06-20 23:08:05 on dist-test-slave-zpss"
server_key: "88af5d939a61864c3f9f29c00186d71d"
server_key_iv: "1298d3edd55588fa088c55d1fd88c598"
server_key_version: "encryptionkey@0"
I20260620 23:08:05.842837 7921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "b534e26442f840ee835cffa6e5befae3"
format_stamp: "Formatted at 2026-06-20 23:08:05 on dist-test-slave-zpss"
server_key: "88af5d939a61864c3f9f29c00186d71d"
server_key_iv: "1298d3edd55588fa088c55d1fd88c598"
server_key_version: "encryptionkey@0"
I20260620 23:08:05.844022 7921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:05.844671 7936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:05.844822 7921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:05.844903 7921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "b534e26442f840ee835cffa6e5befae3"
format_stamp: "Formatted at 2026-06-20 23:08:05 on dist-test-slave-zpss"
server_key: "88af5d939a61864c3f9f29c00186d71d"
server_key_iv: "1298d3edd55588fa088c55d1fd88c598"
server_key_version: "encryptionkey@0"
I20260620 23:08:05.844991 7921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:05.864897 7921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:05.866668 7921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:05.866816 7921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:05.870518 7921 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:42669
I20260620 23:08:05.870564 7988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:42669 every 8 connection(s)
I20260620 23:08:05.870863 7921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:05.871667 7989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:05.873045 7989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:05.875422 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7921
I20260620 23:08:05.875504 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:08:05.875676 18161 external_mini_cluster.cc:1428] Setting key a28577b9b04bac6615b503ea2bacfd37
Jun 20 23:08:05 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:08:05.882691 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:05.873648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42669 (local address 127.0.0.1:45414)
0620 23:08:05.873799 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:05.873839 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:05.874252 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:05.874489 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:05.874491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:05.874720 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:05.874870 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:05.874873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:05.875967 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:05.875968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:05.876626 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:05.876628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:05.877148 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:05.877149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:05.877227 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:05.877778 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:05.877787 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:05.879265 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:05.881302 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:05.881308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:05.881310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:05.881528 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:05.881782 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:05.881782 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:05.881783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:05.881854 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:05.882301 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:05.882306 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:05.882354 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 23:08:05.882439 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 23:08:05.882549 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260620 23:08:05.882774 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:05.873702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.17.188.126:42669)
0620 23:08:05.873887 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:05.873890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:05.873917 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:08:05.874372 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:05.874373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:05.874389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:05.874461 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:05.874948 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:05.875836 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:05.876726 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:05.877051 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:05.877173 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:05.879375 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:05.879385 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:05.879385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:05.879393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:05.881225 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:05.881610 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:05.881613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:05.881614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:05.881711 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:05.881918 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:05.881920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:05.881920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:05.882225 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:05.882307 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:05.882499 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 23:08:05.882627 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":159,"thread_start_us":94,"threads_started":1}
I20260620 23:08:05.883486 7989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.188.125@KRBTEST.COM; defaulting to no policy
I20260620 23:08:05.883826 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:05.883693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:35307 (local address 127.0.0.1:55330)
0620 23:08:05.883745 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:05.883795 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 23:08:05.883968 7940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
I20260620 23:08:05.884338 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:05.884271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:35307 (local address 127.0.0.1:55346)
0620 23:08:05.884298 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:05.884323 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260620 23:08:05.884462 7989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.125:35307: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab.
Entry for principal kudu/127.17.188.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab.
I20260620 23:08:05.899572 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.125:35307
--webserver_interface=127.17.188.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab
--principal=kudu/127.17.188.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42669,127.17.188.125:35307,127.17.188.124:43367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:08:05.933696 7989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } attempt: 1
I20260620 23:08:05.934239 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:05.934027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:35307 (local address 127.0.0.1:55362)
0620 23:08:05.934116 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:05.934190 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260620 23:08:05.934700 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:05.934573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:35307 (local address 127.0.0.1:55364)
0620 23:08:05.934630 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:05.934668 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 23:08:05.934839 7989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.125:35307: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
W20260620 23:08:05.978936 7996 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:08:05.979128 7996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:05.979146 7996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:05.979164 7996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:05.980317 7996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:05.980355 7996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:05.980370 7996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:05.980423 7996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:05.980448 7996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:05.982028 7996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42669,127.17.188.125:35307,127.17.188.124:43367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab
--principal=kudu/127.17.188.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.125:35307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.188.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:05.982268 7996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:05.982491 7996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:05.987088 8001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:05.987140 8002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:05.987251 8004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:05.987262 7996 server_base.cc:1034] running on GCE node
I20260620 23:08:05.987540 7996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:05.987835 7996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:05.989014 7996 hybrid_clock.cc:648] HybridClock initialized: now 1781996885989003 us; error 41 us; skew 500 ppm
Jun 20 23:08:05 dist-test-slave-zpss krb5kdc[7908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:05.991425 7996 init.cc:377] Logged in from keytab as kudu/127.17.188.125@KRBTEST.COM (short username kudu)
I20260620 23:08:05.991878 7996 webserver.cc:460] Webserver started at http://127.17.188.125:44779/ using document root <none> and password file <none>
I20260620 23:08:05.992100 7996 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:05.992174 7996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:05.992323 7996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:05.993376 7996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/data/instance:
uuid: "2c4114f4675a47459f55c842ee8c7bd6"
format_stamp: "Formatted at 2026-06-20 23:08:05 on dist-test-slave-zpss"
server_key: "9b79890576ff792f080bc2d0c840ab6a"
server_key_iv: "168ce1f7e982301b389f13c48e9a5876"
server_key_version: "encryptionkey@0"
I20260620 23:08:05.993747 7996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/wal/instance:
uuid: "2c4114f4675a47459f55c842ee8c7bd6"
format_stamp: "Formatted at 2026-06-20 23:08:05 on dist-test-slave-zpss"
server_key: "9b79890576ff792f080bc2d0c840ab6a"
server_key_iv: "168ce1f7e982301b389f13c48e9a5876"
server_key_version: "encryptionkey@0"
I20260620 23:08:05.995033 7996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:05.995786 8011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:05.995977 7996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:05.996073 7996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/wal
uuid: "2c4114f4675a47459f55c842ee8c7bd6"
format_stamp: "Formatted at 2026-06-20 23:08:05 on dist-test-slave-zpss"
server_key: "9b79890576ff792f080bc2d0c840ab6a"
server_key_iv: "168ce1f7e982301b389f13c48e9a5876"
server_key_version: "encryptionkey@0"
I20260620 23:08:05.996156 7996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:06.003005 7989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } attempt: 2
I20260620 23:08:06.003392 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.003249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:35307 (local address 127.0.0.1:55368)
0620 23:08:06.003313 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.003358 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 23:08:06.003643 7996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:06.003790 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.003657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:35307 (local address 127.0.0.1:55374)
0620 23:08:06.003707 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.003755 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 23:08:06.003913 7989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.125:35307: Network error: Client connection negotiation failed: client connection to 127.17.188.125:35307: connect: Connection refused (error 111)
I20260620 23:08:06.005873 7996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:06.005973 7996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:06.009647 7996 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.125:35307
I20260620 23:08:06.009675 8063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.125:35307 every 8 connection(s)
I20260620 23:08:06.009989 7996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
I20260620 23:08:06.010802 8064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:06.012172 8064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:06.014096 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 7996
I20260620 23:08:06.014195 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-1/wal/instance
I20260620 23:08:06.014366 18161 external_mini_cluster.cc:1428] Setting key b153a32f5cd553052221e8fae26a8140
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:08:06.020799 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.012791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.17.188.126:42669)
0620 23:08:06.012871 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.012873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.012973 (+ 100us) server_negotiation.cc:407] Connection header received
0620 23:08:06.013461 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.013462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.013473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.013520 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.014050 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.014737 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.015667 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.015890 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.015973 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.018004 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.018006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.018006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.018014 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.019465 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.019882 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.019884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.019884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.019951 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.020187 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.020188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.020189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.020298 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.020373 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.020620 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.020707 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 23:08:06.021018 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.012746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42669 (local address 127.0.0.1:45418)
0620 23:08:06.012884 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.012916 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:06.013391 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:06.013567 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:06.013570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:06.013786 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:06.013972 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.013975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.014822 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.014823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.015558 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.015560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.015983 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.015984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.016093 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.016591 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:06.016598 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:06.017892 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:06.019564 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.019571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.019573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.019791 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.020015 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.020016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.020017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.020121 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.020365 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:06.020371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:06.020497 (+ 126us) client_negotiation.cc:769] Sending connection context
0620 23:08:06.020600 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 23:08:06.020745 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.17.188.124@KRBTEST.COM; defaulting to no policy
I20260620 23:08:06.021528 8064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.124@KRBTEST.COM" created.
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
I20260620 23:08:06.028421 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.021836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55386 (local address 127.17.188.125:35307)
0620 23:08:06.021950 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.021954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.021967 (+ 13us) server_negotiation.cc:407] Connection header received
0620 23:08:06.021981 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.021982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.021990 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.022039 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.022296 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.022952 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.023674 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.023974 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.024066 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.025808 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.025815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.025815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.025823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.027242 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.027642 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.027644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.027644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.027685 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.027872 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.027872 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.027872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.027966 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.028014 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.028162 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.028286 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260620 23:08:06.028467 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.021755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:35307 (local address 127.0.0.1:55386)
0620 23:08:06.021798 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.021807 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:06.021840 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:06.022065 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:06.022066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:06.022187 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:06.022230 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.022231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.023021 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.023021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.023596 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.023598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.024067 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.024068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.024131 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.024463 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:06.024468 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:06.025729 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:06.027325 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.027326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.027327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.027566 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.027748 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.027749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.027749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.027814 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.028024 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:06.028025 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:06.028069 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 23:08:06.028143 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 23:08:06.028246 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 23:08:06.029223 8064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:06.029567 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.029445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50462)
0620 23:08:06.029494 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.029543 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 23:08:06.029706 8012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
I20260620 23:08:06.030109 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.030006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50476)
0620 23:08:06.030049 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.030081 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 23:08:06.030215 8064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:43367: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab.
Entry for principal kudu/127.17.188.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab.
I20260620 23:08:06.039984 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.124:43367
--webserver_interface=127.17.188.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab
--principal=kudu/127.17.188.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:42669,127.17.188.125:35307,127.17.188.124:43367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:08:06.079442 8064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } attempt: 1
I20260620 23:08:06.079949 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.079801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50478)
0620 23:08:06.079868 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.079914 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 23:08:06.080293 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.080207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50494)
0620 23:08:06.080250 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.080275 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:08:06.080427 8064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:43367: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
I20260620 23:08:06.095161 7989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } attempt: 3
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
I20260620 23:08:06.102324 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.095559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55394 (local address 127.17.188.125:35307)
0620 23:08:06.095620 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.095623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.095633 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:08:06.095712 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.095714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.095726 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.095798 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.096109 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.096607 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.097366 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.097592 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.097670 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.099837 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.099838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.099839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.099846 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.101160 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.101546 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.101548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.101548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.101602 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.101795 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.101795 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.101796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.101910 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.101982 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.102136 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.102229 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:08:06.102324 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.095491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:35307 (local address 127.0.0.1:55394)
0620 23:08:06.095545 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.095560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:06.095624 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:06.095803 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:06.095804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:06.095964 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:06.096026 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.096028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.096705 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.096707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.097250 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.097252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.097681 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.097682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.097760 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.098276 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:06.098280 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:06.099737 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:06.101247 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.101248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.101248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.101463 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.101669 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.101670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.101670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.101726 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.101987 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:06.101988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:06.102031 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 23:08:06.102115 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 23:08:06.102228 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 23:08:06.102947 7989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:06.103317 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.103219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50500)
0620 23:08:06.103266 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.103295 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 23:08:06.103693 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.103577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50508)
0620 23:08:06.103626 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.103660 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 23:08:06.103839 7989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:43367: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
W20260620 23:08:06.121752 8071 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:08:06.121924 8071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:06.121940 8071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:06.121994 8071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:06.123136 8071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:06.123174 8071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:06.123188 8071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:06.123235 8071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:06.123260 8071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:06.124734 8071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:42669,127.17.188.125:35307,127.17.188.124:43367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab
--principal=kudu/127.17.188.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.124:43367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.188.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:06.124961 8071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:06.125169 8071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:06.129765 8076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:06.129828 8079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:06.129944 8077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:06.130159 8071 server_base.cc:1034] running on GCE node
I20260620 23:08:06.130394 8071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:06.130627 8071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:06.131785 8071 hybrid_clock.cc:648] HybridClock initialized: now 1781996886131770 us; error 25 us; skew 500 ppm
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:06.134176 8071 init.cc:377] Logged in from keytab as kudu/127.17.188.124@KRBTEST.COM (short username kudu)
I20260620 23:08:06.134618 8071 webserver.cc:460] Webserver started at http://127.17.188.124:34773/ using document root <none> and password file <none>
I20260620 23:08:06.134799 8071 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:06.134837 8071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:06.134919 8071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:06.134989 7989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } attempt: 1
I20260620 23:08:06.135327 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.135216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50512)
0620 23:08:06.135270 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.135301 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 23:08:06.135641 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.135587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50522)
0620 23:08:06.135608 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.135628 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260620 23:08:06.135753 7989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:43367: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
I20260620 23:08:06.136113 8071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/data/instance:
uuid: "e3b7c493bb2e44aca858222ae3450470"
format_stamp: "Formatted at 2026-06-20 23:08:06 on dist-test-slave-zpss"
server_key: "8f296cc69cc48ef1084c881c5cf47bff"
server_key_iv: "ab4dd1cc6ec86bcc0967c2ad9a2e58ac"
server_key_version: "encryptionkey@0"
I20260620 23:08:06.136389 8071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/wal/instance:
uuid: "e3b7c493bb2e44aca858222ae3450470"
format_stamp: "Formatted at 2026-06-20 23:08:06 on dist-test-slave-zpss"
server_key: "8f296cc69cc48ef1084c881c5cf47bff"
server_key_iv: "ab4dd1cc6ec86bcc0967c2ad9a2e58ac"
server_key_version: "encryptionkey@0"
I20260620 23:08:06.137650 8071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:08:06.138319 8086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:06.138499 8071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:06.138566 8071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/wal
uuid: "e3b7c493bb2e44aca858222ae3450470"
format_stamp: "Formatted at 2026-06-20 23:08:06 on dist-test-slave-zpss"
server_key: "8f296cc69cc48ef1084c881c5cf47bff"
server_key_iv: "ab4dd1cc6ec86bcc0967c2ad9a2e58ac"
server_key_version: "encryptionkey@0"
I20260620 23:08:06.138659 8071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:06.148552 8064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } attempt: 2
I20260620 23:08:06.148911 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.148801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50526)
0620 23:08:06.148853 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.148885 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 23:08:06.149228 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.149147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50534)
0620 23:08:06.149184 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.149213 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 23:08:06.149354 8064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:43367: Network error: Client connection negotiation failed: client connection to 127.17.188.124:43367: connect: Connection refused (error 111)
I20260620 23:08:06.157033 8071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:06.159233 8071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:06.159381 8071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:06.163216 8071 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.124:43367
I20260620 23:08:06.163242 8138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.124:43367 every 8 connection(s)
I20260620 23:08:06.163576 8071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
I20260620 23:08:06.164284 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 8071
I20260620 23:08:06.164386 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996599092328-18161-0/minicluster-data/master-2/wal/instance
I20260620 23:08:06.164517 8139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:06.164602 18161 external_mini_cluster.cc:1428] Setting key a50346ecb6eea4db2266a23676de51d5
I20260620 23:08:06.166961 8139 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:08:06.174954 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.167553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45428 (local address 127.17.188.126:42669)
0620 23:08:06.167614 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.167616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.167801 (+ 185us) server_negotiation.cc:407] Connection header received
0620 23:08:06.167850 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.167851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.167866 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.167907 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.168409 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.168991 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.169932 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.170109 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.170178 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.172221 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.172223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.172223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.172230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.173513 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.174129 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.174130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.174131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.174188 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.174400 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.174401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.174401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.174507 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.174594 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.174781 (+ 187us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.174853 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:08:06.174993 8142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.165071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.17.188.124:43367)
0620 23:08:06.165237 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.165249 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.165273 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:08:06.165712 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.165718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.165761 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.165867 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.166592 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.167700 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.168689 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.169252 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.169369 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.171739 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.171746 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.171746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.171753 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.173511 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.173838 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.173840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.173841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.173914 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.174132 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.174133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.174133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.174338 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.174491 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.174610 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.174849 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260620 23:08:06.175019 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.167496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:42669 (local address 127.0.0.1:45428)
0620 23:08:06.167678 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.167707 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:06.167790 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:06.167942 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:06.167944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:06.168266 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:06.168335 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.168337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.169090 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.169091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.169841 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.169844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.170211 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.170212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.170302 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.170704 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:06.170715 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:06.172132 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:06.173843 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.173848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.173850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.174037 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.174255 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.174256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.174257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.174313 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.174585 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:06.174592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:06.174652 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 23:08:06.174744 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 23:08:06.174910 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":56,"thread_start_us":74,"threads_started":1}
I20260620 23:08:06.175478 8139 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:08:06.182118 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.175698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:35307 (local address 127.0.0.1:55396)
0620 23:08:06.175793 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.175804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:06.175848 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:06.175963 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:06.175963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:06.176089 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:06.176127 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.176128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.176714 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.176715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.177162 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.177163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.177591 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.177592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.177657 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.178052 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:06.178057 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:06.179547 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:06.181011 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.181012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.181013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.181218 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.181474 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.181475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.181475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.181555 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.181811 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:06.181812 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:06.181855 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 23:08:06.181907 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 23:08:06.182007 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260620 23:08:06.182152 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.175756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.17.188.125:35307)
0620 23:08:06.175797 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.175799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.175835 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:08:06.175898 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.175899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.175908 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.175959 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.176205 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.176636 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.177243 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.177521 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.177594 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.179629 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.179630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.179631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.179638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.180934 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.181299 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.181300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.181300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.181391 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.181630 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.181631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.181631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.181736 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.181818 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.181956 (+ 138us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.182030 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:06.182826 8139 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:06.183213 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.175765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.17.188.126:42669)
0620 23:08:06.175798 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.175800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.175847 (+ 47us) server_negotiation.cc:407] Connection header received
0620 23:08:06.175914 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.175915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.175923 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.175975 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.176404 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.176816 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.177520 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.177700 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.177777 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.179729 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.179730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.179730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.179735 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.181454 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.181813 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.181814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.181815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.181865 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.182084 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.182085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.182086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.182206 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.182820 (+ 614us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.182853 (+ 33us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.182917 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
I20260620 23:08:06.189224 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.183048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50538)
0620 23:08:06.183093 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.183103 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:06.183138 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:06.183248 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:06.183248 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:06.183402 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:06.183433 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.183434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.184229 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.184230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.184793 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.184796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.185178 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.185179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.185239 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.185570 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:06.185575 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:06.186769 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:06.188270 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.188270 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.188271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.188442 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.188637 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.188638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.188638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.188689 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.188928 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:06.188929 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:06.188965 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 23:08:06.189029 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:08:06.189120 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 23:08:06.189286 8142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.183096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.17.188.124:43367)
0620 23:08:06.183126 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.183128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.183135 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:06.183192 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.183193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.183201 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.183243 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.183515 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.184151 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.184866 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.185111 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.185156 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.186846 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.186847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.186847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.186854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.188160 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.188523 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.188524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.188525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.188575 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.188765 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.188766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.188767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.188864 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.188936 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.189071 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.189167 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":14}
I20260620 23:08:06.190449 8139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470: Bootstrap starting.
I20260620 23:08:06.191012 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.183806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55406 (local address 127.17.188.125:35307)
0620 23:08:06.183839 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.183841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.183848 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:06.183866 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.183866 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.183882 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.183941 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.184428 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.184864 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.185671 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.185852 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.185910 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.188224 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.188225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.188225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.188245 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.189905 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.190244 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.190245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.190245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.190300 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.190499 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.190500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.190500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.190606 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.190802 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.190843 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.190920 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:08:06.191478 8139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:06.191815 8139 log.cc:824] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:06.192397 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:06.192680 8139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470: No bootstrap required, opened a new log
I20260620 23:08:06.193248 7963 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b534e26442f840ee835cffa6e5befae3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45430
I20260620 23:08:06.194280 8139 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } }
I20260620 23:08:06.194418 8139 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b7c493bb2e44aca858222ae3450470, State: Initialized, Role: FOLLOWER
I20260620 23:08:06.194525 8139 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } }
I20260620 23:08:06.194808 8139 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:06.194796 8145 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.194905 8145 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:08:06.196524 8156 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:06.196805 8156 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:06.196843 8071 master_runner.cc:186] Error getting master registration for 127.17.188.126:42669: OK, instance_id { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" instance_seqno: 1781996885869442 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:08:06.199440 8071 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:08:06.211043 7989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } attempt: 2
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
I20260620 23:08:06.218904 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.211379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50552)
0620 23:08:06.211430 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.211442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:06.211489 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:06.211665 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:06.211665 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:06.211871 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:06.211933 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.211936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.212833 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.212834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.213676 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.213678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.214123 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.214124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.214176 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.214661 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:06.214666 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:06.215922 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:06.217748 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.217749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.217749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.218018 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.218237 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.218238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.218238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.218300 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.218597 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:06.218598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:06.218640 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 23:08:06.218707 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 23:08:06.218798 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260620 23:08:06.218907 8142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.211461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50552 (local address 127.17.188.124:43367)
0620 23:08:06.211519 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.211521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.211530 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:06.211551 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.211552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.211561 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.211627 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.212010 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.212752 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.213756 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.214045 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.214116 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.216006 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.216007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.216007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.216014 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.217654 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.218108 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.218109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.218109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.218168 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.218364 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.218365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.218366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.218530 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.218599 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.218745 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.218820 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:08:06.220448 7989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3: Bootstrap starting.
I20260620 23:08:06.221122 7989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:06.221446 7989 log.cc:824] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:06.222187 7989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3: No bootstrap required, opened a new log
I20260620 23:08:06.223171 7989 raft_consensus.cc:348] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } }
I20260620 23:08:06.223301 7989 raft_consensus.cc:729] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b534e26442f840ee835cffa6e5befae3, State: Initialized, Role: FOLLOWER
I20260620 23:08:06.223388 7989 consensus_queue.cc:260] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } }
I20260620 23:08:06.223631 8159 sys_catalog.cc:455] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.223690 7989 sys_catalog.cc:564] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:06.223721 7963 raft_consensus.cc:482] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 23:08:06.223757 8159 sys_catalog.cc:458] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:06.223816 7963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:06.224615 7963 raft_consensus.cc:504] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } }
I20260620 23:08:06.224838 7963 leader_election.cc:288] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [CANDIDATE]: Term 1 election: Requested vote from peers 2c4114f4675a47459f55c842ee8c7bd6 (127.17.188.125:35307), e3b7c493bb2e44aca858222ae3450470 (127.17.188.124:43367)
I20260620 23:08:06.225505 8113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b534e26442f840ee835cffa6e5befae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e3b7c493bb2e44aca858222ae3450470"
I20260620 23:08:06.225625 8113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:06.226379 8113 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b534e26442f840ee835cffa6e5befae3 in term 1.
I20260620 23:08:06.227308 7937 leader_election.cc:302] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b534e26442f840ee835cffa6e5befae3, e3b7c493bb2e44aca858222ae3450470; no voters:
I20260620 23:08:06.227437 8159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:06.227560 8159 raft_consensus.cc:686] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [term 1 LEADER]: Becoming Leader. State: Replica: b534e26442f840ee835cffa6e5befae3, State: Running, Role: LEADER
I20260620 23:08:06.227643 8159 consensus_queue.cc:237] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } }
I20260620 23:08:06.228024 8159 sys_catalog.cc:455] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b534e26442f840ee835cffa6e5befae3. Latest consensus state: current_term: 1 leader_uuid: "b534e26442f840ee835cffa6e5befae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.228096 8159 sys_catalog.cc:458] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [sys.catalog]: This master's current role is: LEADER
W20260620 23:08:06.228780 8170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:06.228870 8170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:06.226318 7921 master_runner.cc:186] Error getting master registration for 127.17.188.125:35307: OK, instance_id { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" instance_seqno: 1781996886008597 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:08:06.229230 7921 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:08:06.229761 8179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:06.229933 8179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:06.231508 8113 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [term 1 FOLLOWER]: Refusing update from remote peer b534e26442f840ee835cffa6e5befae3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:06.231935 8175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:06.234534 8145 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b534e26442f840ee835cffa6e5befae3. Latest consensus state: current_term: 1 leader_uuid: "b534e26442f840ee835cffa6e5befae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.234616 8145 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:06.236126 8159 sys_catalog.cc:455] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b534e26442f840ee835cffa6e5befae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.236212 8159 sys_catalog.cc:458] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:06.236698 8145 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b534e26442f840ee835cffa6e5befae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.236786 8145 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:06.237120 8181 mvcc.cc:205] Tried to move back new op lower bound from 7299059246001909760 to 7299059245989097472. Current Snapshot: MvccSnapshot[applied={T|T < 7299059246001909760}]
I20260620 23:08:06.238025 8183 mvcc.cc:205] Tried to move back new op lower bound from 7299059246001909760 to 7299059245989097472. Current Snapshot: MvccSnapshot[applied={T|T < 7299059246001909760}]
I20260620 23:08:06.240495 8064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } attempt: 3
I20260620 23:08:06.240715 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.227851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55418 (local address 127.17.188.125:35307)
0620 23:08:06.227889 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.227891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.228760 (+ 869us) server_negotiation.cc:407] Connection header received
0620 23:08:06.228962 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.228963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.229002 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.229031 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.229632 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.230325 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.232725 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.233022 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.233064 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.236308 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.236309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.236310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.236317 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.237936 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.239862 (+ 1926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.239863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.239863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.239919 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.240128 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.240129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.240129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.240252 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.240330 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.240544 (+ 214us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.240622 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:06.241330 8179 catalog_manager.cc:1347] Generated new cluster ID: 949aef35abdd4ac992d506938bbf6198
I20260620 23:08:06.241379 8179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:06.242234 8142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.227412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.17.188.124:43367)
0620 23:08:06.227455 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.227457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.228230 (+ 773us) server_negotiation.cc:407] Connection header received
0620 23:08:06.228365 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.228365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.228395 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.228424 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.230161 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.230863 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.233572 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.233872 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.233913 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.238096 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.238097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.238097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.238105 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.239746 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.240769 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.240770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.240771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.240829 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.241579 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.241580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.241580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.241698 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.241775 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.242072 (+ 297us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.242144 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Jun 20 23:08:06 dist-test-slave-zpss krb5kdc[7908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
I20260620 23:08:06.249413 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.240881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:43367 (local address 127.0.0.1:50566)
0620 23:08:06.240984 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:06.240993 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:06.241028 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:06.241884 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:06.241885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:06.242051 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:06.242088 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.242089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.243375 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.243376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.244114 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:06.244116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.244600 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.244600 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:06.244658 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.245084 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:06.245101 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:06.246496 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:06.248308 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.248309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.248310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.248519 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.248757 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:06.248758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:06.248758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:06.248824 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:06.249088 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:06.249088 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:06.249137 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 23:08:06.249193 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 23:08:06.249267 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260620 23:08:06.249444 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.241093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50566 (local address 127.17.188.124:43367)
0620 23:08:06.241756 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.241758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.241765 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:06.241782 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.241783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.241792 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.242178 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.242206 (+ 28us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.243288 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.244193 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.244510 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.244583 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.246587 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.246588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.246589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.246595 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.248188 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.248589 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.248590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.248590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.248649 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.248882 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.248883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.248883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.249003 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.249073 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.249232 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.249305 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":73,"threads_started":1}
I20260620 23:08:06.250759 8064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6: Bootstrap starting.
I20260620 23:08:06.251415 8064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:06.251698 8064 log.cc:824] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:06.252570 8064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6: No bootstrap required, opened a new log
I20260620 23:08:06.254542 8064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } }
I20260620 23:08:06.254668 8064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4114f4675a47459f55c842ee8c7bd6, State: Initialized, Role: FOLLOWER
I20260620 23:08:06.254737 8064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } }
I20260620 23:08:06.254997 8064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:06.255119 8038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b534e26442f840ee835cffa6e5befae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2c4114f4675a47459f55c842ee8c7bd6"
I20260620 23:08:06.255229 8038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:06.256013 8038 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b534e26442f840ee835cffa6e5befae3 in term 1.
W20260620 23:08:06.256572 8017 tablet.cc:2369] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:06.256282 8037 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [term 1 FOLLOWER]: Refusing update from remote peer b534e26442f840ee835cffa6e5befae3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:06.257714 8186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:06.258724 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:06.230680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.17.188.126:42669)
0620 23:08:06.230714 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.230716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.236012 (+ 5296us) server_negotiation.cc:407] Connection header received
0620 23:08:06.236114 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.236115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.236124 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.236154 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.236790 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.237509 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.238488 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:06.238778 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:06.238816 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:06.253979 (+ 15163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:06.253981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:06.253981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:06.254000 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:06.257427 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.257799 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.257800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.257801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.257871 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:06.258121 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:06.258122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:06.258122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:06.258246 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:06.258313 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:06.258532 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 23:08:06.258615 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:06.259668 8205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b534e26442f840ee835cffa6e5befae3. Latest consensus state: current_term: 1 leader_uuid: "b534e26442f840ee835cffa6e5befae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.259794 8205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:06.259863 8205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b534e26442f840ee835cffa6e5befae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.259924 8205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:08:06.260259 8201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:06.260311 8201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:06.260469 8190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b534e26442f840ee835cffa6e5befae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.260526 8190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:06.270107 8203 mvcc.cc:205] Tried to move back new op lower bound from 7299059246001909760 to 7299059245989097472. Current Snapshot: MvccSnapshot[applied={T|T < 7299059246001909760}]
I20260620 23:08:06.275396 8186 sys_catalog.cc:455] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b534e26442f840ee835cffa6e5befae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b534e26442f840ee835cffa6e5befae3" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 42669 } } peers { permanent_uuid: "2c4114f4675a47459f55c842ee8c7bd6" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 35307 } } peers { permanent_uuid: "e3b7c493bb2e44aca858222ae3450470" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 43367 } } }
I20260620 23:08:06.275481 8186 sys_catalog.cc:458] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:06.277603 8179 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:06.278349 8179 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:06.287848 8179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b534e26442f840ee835cffa6e5befae3: Generated new TSK 0
I20260620 23:08:06.288003 8179 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:08:06.310276 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.309037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55432 (local address 127.17.188.125:35307)
0620 23:08:06.309078 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.309080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.309683 (+ 603us) server_negotiation.cc:407] Connection header received
0620 23:08:06.309780 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.309781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.309792 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.309821 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.310216 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55432: BlockingRecv error: recv got EOF from 127.0.0.1:55432 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:08:06.310968 7992 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.309423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.17.188.126:42669)
0620 23:08:06.309458 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.309460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.310496 (+ 1036us) server_negotiation.cc:407] Connection header received
0620 23:08:06.310586 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.310587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.310596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.310623 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.310904 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45438: BlockingRecv error: recv got EOF from 127.0.0.1:45438 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 23:08:06.311882 8188 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.311272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50580 (local address 127.17.188.124:43367)
0620 23:08:06.311308 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.311309 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.311415 (+ 106us) server_negotiation.cc:407] Connection header received
0620 23:08:06.311502 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.311503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.311512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.311541 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.311815 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50580: BlockingRecv error: recv got EOF from 127.0.0.1:50580 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 23:08:06.314039 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.313376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.17.188.125:35307)
0620 23:08:06.313408 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.313409 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.313575 (+ 166us) server_negotiation.cc:407] Connection header received
0620 23:08:06.313669 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.313670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.313684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.313711 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.313999 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55444: BlockingRecv error: recv got EOF from 127.0.0.1:55444 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:08:06.316936 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.316185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55460 (local address 127.17.188.125:35307)
0620 23:08:06.316214 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.316216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.316449 (+ 233us) server_negotiation.cc:407] Connection header received
0620 23:08:06.316537 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.316538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.316547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.316573 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.316896 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55460: BlockingRecv error: recv got EOF from 127.0.0.1:55460 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:08:06.320619 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.319936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.17.188.125:35307)
0620 23:08:06.319967 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.319969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.320179 (+ 210us) server_negotiation.cc:407] Connection header received
0620 23:08:06.320267 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.320268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.320276 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.320303 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.320579 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55474: BlockingRecv error: recv got EOF from 127.0.0.1:55474 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:08:06.326364 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.325652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.17.188.125:35307)
0620 23:08:06.325682 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.325684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.325924 (+ 240us) server_negotiation.cc:407] Connection header received
0620 23:08:06.326013 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.326014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.326023 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.326050 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.326325 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55480: BlockingRecv error: recv got EOF from 127.0.0.1:55480 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 23:08:06.336113 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.335338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.17.188.125:35307)
0620 23:08:06.335370 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.335372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.335629 (+ 257us) server_negotiation.cc:407] Connection header received
0620 23:08:06.335722 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.335723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.335732 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.335759 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.336071 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55494: BlockingRecv error: recv got EOF from 127.0.0.1:55494 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 23:08:06.354166 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.353280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.17.188.125:35307)
0620 23:08:06.353341 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.353343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.353619 (+ 276us) server_negotiation.cc:407] Connection header received
0620 23:08:06.353719 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.353720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.353730 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.353759 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.354114 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55502: BlockingRecv error: recv got EOF from 127.0.0.1:55502 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:08:06.388403 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.387473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55516 (local address 127.17.188.125:35307)
0620 23:08:06.387519 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.387521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.387798 (+ 277us) server_negotiation.cc:407] Connection header received
0620 23:08:06.387926 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.387927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.387940 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.387970 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.388343 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55516: BlockingRecv error: recv got EOF from 127.0.0.1:55516 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 23:08:06.454511 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.453776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.17.188.125:35307)
0620 23:08:06.453851 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.453853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.454008 (+ 155us) server_negotiation.cc:407] Connection header received
0620 23:08:06.454057 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.454057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.454067 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.454118 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.454427 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55524: BlockingRecv error: recv got EOF from 127.0.0.1:55524 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260620 23:08:06.584227 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.583569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55540 (local address 127.17.188.125:35307)
0620 23:08:06.583671 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.583673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.583717 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:08:06.583776 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.583777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.583800 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.583842 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.584164 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55540: BlockingRecv error: recv got EOF from 127.0.0.1:55540 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260620 23:08:06.842017 8068 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:06.841465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55546 (local address 127.17.188.125:35307)
0620 23:08:06.841509 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:06.841512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:06.841526 (+ 14us) server_negotiation.cc:407] Connection header received
0620 23:08:06.841619 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:06.841620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:06.841629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:06.841679 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:06.841971 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55546: BlockingRecv error: recv got EOF from 127.0.0.1:55546 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:08:07.197278 8156 catalog_manager.cc:1259] Loaded cluster ID: 949aef35abdd4ac992d506938bbf6198
I20260620 23:08:07.197373 8156 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470: loading cluster ID for follower catalog manager: success
I20260620 23:08:07.198154 8156 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470: acquiring CA information for follower catalog manager: success
I20260620 23:08:07.198491 8156 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3b7c493bb2e44aca858222ae3450470: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:08:07.260679 8201 catalog_manager.cc:1259] Loaded cluster ID: 949aef35abdd4ac992d506938bbf6198
I20260620 23:08:07.260776 8201 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6: loading cluster ID for follower catalog manager: success
I20260620 23:08:07.261570 8201 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6: acquiring CA information for follower catalog manager: success
I20260620 23:08:07.261888 8201 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2c4114f4675a47459f55c842ee8c7bd6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:08:07.358067 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:07.355110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55554 (local address 127.17.188.125:35307)
0620 23:08:07.355226 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:07.355228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:07.355319 (+ 91us) server_negotiation.cc:407] Connection header received
0620 23:08:07.355400 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:07.355401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:07.355494 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:07.355609 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:07.355761 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.356324 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.357164 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.357406 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.357481 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:07.357694 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:07.357805 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:07.357919 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:07.357959 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 23:08:07.357978 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":64,"threads_started":1}
I20260620 23:08:07.362001 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:07.359741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50588 (local address 127.17.188.124:43367)
0620 23:08:07.359849 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:07.359851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:07.359888 (+ 37us) server_negotiation.cc:407] Connection header received
0620 23:08:07.359962 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:07.359963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:07.359994 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:07.360049 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:07.360128 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.360585 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.361263 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.361516 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.361566 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:07.361701 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:07.361782 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:07.361835 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:07.361875 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 23:08:07.361890 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260620 23:08:07.366660 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:07.363531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55560 (local address 127.17.188.125:35307)
0620 23:08:07.363576 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:07.363578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:07.363661 (+ 83us) server_negotiation.cc:407] Connection header received
0620 23:08:07.363742 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:07.363743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:07.363752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:07.363812 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:07.363935 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.364528 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.365520 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.365809 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.365897 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:07.366413 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:07.366442 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:07.366502 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:07.366556 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 23:08:07.366573 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 23:08:07.366729 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:07.363576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45444 (local address 127.17.188.126:42669)
0620 23:08:07.363717 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:07.363719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:07.363726 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:07.363769 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:07.363769 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:07.363817 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:07.363915 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:07.364013 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.364786 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.365762 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.366071 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.366139 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:07.366313 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:07.366418 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:07.366482 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:07.366628 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 23:08:07.366652 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":53,"threads_started":1}
I20260620 23:08:07.366902 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:07.363540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50596 (local address 127.17.188.124:43367)
0620 23:08:07.363669 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:07.363671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:07.364043 (+ 372us) server_negotiation.cc:407] Connection header received
0620 23:08:07.364095 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:07.364096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:07.364106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:07.364154 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:07.364273 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.364943 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.365901 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:07.366174 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:07.366228 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:07.366372 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:07.366403 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:07.366465 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:07.366828 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 23:08:07.366851 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260620 23:08:07.368083 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7921
I20260620 23:08:07.373397 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 7996
I20260620 23:08:07.378227 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 8071
2026-06-20T23:08:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2763 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:07 dist-test-slave-zpss krb5kdc[8288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:07 dist-test-slave-zpss krb5kdc[8288](info): set up 2 sockets
Jun 20 23:08:07 dist-test-slave-zpss krb5kdc[8288](info): commencing operation
krb5kdc: starting...
W20260620 23:08:08.409583 18161 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/test-user.keytab.
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.188.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
Entry for principal kudu/127.17.188.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab.
I20260620 23:08:08.519892 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34165
--webserver_interface=127.17.188.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:34165,127.17.188.125:38533,127.17.188.124:38513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:08.598067 8301 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:08:08.598292 8301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:08.598320 8301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:08.598363 8301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:08.599579 8301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:08.599663 8301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:08.599689 8301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:08.599720 8301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:08.599758 8301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:08.601241 8301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:34165,127.17.188.125:38533,127.17.188.124:38513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.126.keytab
--principal=kudu/127.17.188.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.126:34165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.188.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:08.601547 8301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:08.601768 8301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:08.606540 8309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:08.606551 8307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:08.606551 8306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:08.606771 8301 server_base.cc:1034] running on GCE node
I20260620 23:08:08.607033 8301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:08.607300 8301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:08.608471 8301 hybrid_clock.cc:648] HybridClock initialized: now 1781996888608457 us; error 26 us; skew 500 ppm
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:08.610865 8301 init.cc:377] Logged in from keytab as kudu/127.17.188.126@KRBTEST.COM (short username kudu)
I20260620 23:08:08.611317 8301 webserver.cc:460] Webserver started at http://127.17.188.126:36917/ using document root <none> and password file <none>
I20260620 23:08:08.611527 8301 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:08.611611 8301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:08.611756 8301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:08.612777 8301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/data/instance:
uuid: "ffd9c9aa318547db964bfbbbb5a37434"
format_stamp: "Formatted at 2026-06-20 23:08:08 on dist-test-slave-zpss"
server_key: "d537e21c350d1c852a5ced5da8a82fd6"
server_key_iv: "b0cc2bfce1f42a610dd4707acee8bd34"
server_key_version: "encryptionkey@0"
I20260620 23:08:08.613132 8301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/wal/instance:
uuid: "ffd9c9aa318547db964bfbbbb5a37434"
format_stamp: "Formatted at 2026-06-20 23:08:08 on dist-test-slave-zpss"
server_key: "d537e21c350d1c852a5ced5da8a82fd6"
server_key_iv: "b0cc2bfce1f42a610dd4707acee8bd34"
server_key_version: "encryptionkey@0"
I20260620 23:08:08.614461 8301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:08.615162 8316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:08.615336 8301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:08.615433 8301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/wal
uuid: "ffd9c9aa318547db964bfbbbb5a37434"
format_stamp: "Formatted at 2026-06-20 23:08:08 on dist-test-slave-zpss"
server_key: "d537e21c350d1c852a5ced5da8a82fd6"
server_key_iv: "b0cc2bfce1f42a610dd4707acee8bd34"
server_key_version: "encryptionkey@0"
I20260620 23:08:08.615516 8301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:08.627470 8301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:08.629818 8301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:08.629987 8301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:08.633814 8301 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.126:34165
I20260620 23:08:08.633872 8368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.126:34165 every 8 connection(s)
I20260620 23:08:08.634210 8301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:08.634992 8369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:08.636418 8369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:08:08.644043 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 8301
I20260620 23:08:08.644125 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-0/wal/instance
I20260620 23:08:08.644367 18161 external_mini_cluster.cc:1428] Setting key ff1dc8361f2736af0076c777828205fc
I20260620 23:08:08.645308 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.637034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49536 (local address 127.17.188.126:34165)
0620 23:08:08.637183 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.637191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.637305 (+ 114us) server_negotiation.cc:407] Connection header received
0620 23:08:08.637850 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.637852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.637872 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.637976 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.638513 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.639330 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.640160 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.640446 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.640584 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.642489 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.642496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.642496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.642503 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.643784 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.644179 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.644182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.644182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.644261 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.644588 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.644588 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.644589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.644775 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.644838 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.645086 (+ 248us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.645171 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260620 23:08:08.645470 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.636968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34165 (local address 127.0.0.1:49536)
0620 23:08:08.637170 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.637210 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:08.637691 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:08.637981 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:08.637983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:08.638261 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:08.638412 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.638417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.639437 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.639438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.640069 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.640070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.640557 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.640558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.640635 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.641036 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:08.641043 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:08.642393 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:08.643889 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.643895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.643897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.644100 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.644353 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.644354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.644355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.644488 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.644853 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:08.644860 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:08.644939 (+ 79us) client_negotiation.cc:769] Sending connection context
0620 23:08:08.645054 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 23:08:08.645271 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":55,"thread_start_us":77,"threads_started":1}
I20260620 23:08:08.646452 8369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:08.646754 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.646646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:38533 (local address 127.0.0.1:37036)
0620 23:08:08.646693 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.646736 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 23:08:08.646878 8319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
I20260620 23:08:08.647266 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.647166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:38533 (local address 127.0.0.1:37052)
0620 23:08:08.647211 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.647233 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 23:08:08.647361 8369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.125:38533: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.188.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab.
Entry for principal kudu/127.17.188.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab.
I20260620 23:08:08.665933 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.125:38533
--webserver_interface=127.17.188.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab
--principal=kudu/127.17.188.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:34165,127.17.188.125:38533,127.17.188.124:38513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:08:08.696592 8369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } attempt: 1
I20260620 23:08:08.697163 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.696983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:38533 (local address 127.0.0.1:37058)
0620 23:08:08.697075 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.697117 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260620 23:08:08.697616 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.697512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:38533 (local address 127.0.0.1:37072)
0620 23:08:08.697566 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.697596 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 23:08:08.697731 8369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.125:38533: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
W20260620 23:08:08.743469 8376 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:08:08.743671 8376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:08.743688 8376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:08.743707 8376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:08.744771 8376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:08.744804 8376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:08.744817 8376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:08.744830 8376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:08.744849 8376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:08.746225 8376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:34165,127.17.188.125:38533,127.17.188.124:38513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.125.keytab
--principal=kudu/127.17.188.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.125:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.188.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:08.746480 8376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:08.746651 8376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:08.751461 8381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:08.751470 8384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:08.751484 8382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:08.751523 8376 server_base.cc:1034] running on GCE node
I20260620 23:08:08.751858 8376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:08.752193 8376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:08.753382 8376 hybrid_clock.cc:648] HybridClock initialized: now 1781996888753366 us; error 26 us; skew 500 ppm
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:08.755774 8376 init.cc:377] Logged in from keytab as kudu/127.17.188.125@KRBTEST.COM (short username kudu)
I20260620 23:08:08.756193 8376 webserver.cc:460] Webserver started at http://127.17.188.125:38003/ using document root <none> and password file <none>
I20260620 23:08:08.756413 8376 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:08.756484 8376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:08.756645 8376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:08.757789 8376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/data/instance:
uuid: "71efd678cbcd4c63a1599d322b322ca1"
format_stamp: "Formatted at 2026-06-20 23:08:08 on dist-test-slave-zpss"
server_key: "c61a84a66c78db2697f2d23d37225958"
server_key_iv: "27b3410ef4d9cc9b6de46d0e1e402e51"
server_key_version: "encryptionkey@0"
I20260620 23:08:08.758157 8376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/wal/instance:
uuid: "71efd678cbcd4c63a1599d322b322ca1"
format_stamp: "Formatted at 2026-06-20 23:08:08 on dist-test-slave-zpss"
server_key: "c61a84a66c78db2697f2d23d37225958"
server_key_iv: "27b3410ef4d9cc9b6de46d0e1e402e51"
server_key_version: "encryptionkey@0"
I20260620 23:08:08.759460 8376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:08.760219 8391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:08.760411 8376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:08.760512 8376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/wal
uuid: "71efd678cbcd4c63a1599d322b322ca1"
format_stamp: "Formatted at 2026-06-20 23:08:08 on dist-test-slave-zpss"
server_key: "c61a84a66c78db2697f2d23d37225958"
server_key_iv: "27b3410ef4d9cc9b6de46d0e1e402e51"
server_key_version: "encryptionkey@0"
I20260620 23:08:08.760592 8376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:08.765865 8369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } attempt: 2
I20260620 23:08:08.766206 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.766089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:38533 (local address 127.0.0.1:37086)
0620 23:08:08.766143 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.766177 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 23:08:08.766577 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.766498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:38533 (local address 127.0.0.1:37092)
0620 23:08:08.766535 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.766559 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 23:08:08.766781 8369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.125:38533: Network error: Client connection negotiation failed: client connection to 127.17.188.125:38533: connect: Connection refused (error 111)
I20260620 23:08:08.774217 8376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:08.776245 8376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:08.776407 8376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:08.780586 8376 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.125:38533
I20260620 23:08:08.780635 8443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.125:38533 every 8 connection(s)
I20260620 23:08:08.780992 8376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/data/info.pb
I20260620 23:08:08.781890 8444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:08.783401 8444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:08:08.790175 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 8376
I20260620 23:08:08.790287 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-1/wal/instance
I20260620 23:08:08.790503 18161 external_mini_cluster.cc:1428] Setting key ec30ae8c4652f10cbdd8f8171d087372
I20260620 23:08:08.792769 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.784151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49548 (local address 127.17.188.126:34165)
0620 23:08:08.784212 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.784214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.784425 (+ 211us) server_negotiation.cc:407] Connection header received
0620 23:08:08.784863 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.784864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.784877 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.784934 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.785587 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.786066 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.786977 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.787180 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.787235 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.789516 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.789518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.789519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.789526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.791329 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.791710 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.791711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.791712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.791774 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.792095 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.792096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.792096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.792246 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.792302 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.792637 (+ 335us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.792705 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:08:08.793042 8446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.784125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34165 (local address 127.0.0.1:49548)
0620 23:08:08.784288 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.784322 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:08.784766 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:08.785008 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:08.785011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:08.785253 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:08.785509 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.785514 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.786141 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.786142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.786905 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.786907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.787252 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.787252 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.787348 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.787920 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:08.787928 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:08.789414 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:08.791420 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.791426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.791428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.791635 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.791843 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.791843 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.791844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.791988 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.792316 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:08.792326 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:08.792495 (+ 169us) client_negotiation.cc:769] Sending connection context
0620 23:08:08.792615 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 23:08:08.792781 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260620 23:08:08.793656 8444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.188.124@KRBTEST.COM; defaulting to no policy
I20260620 23:08:08.800959 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.793963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.17.188.125:38533)
0620 23:08:08.794119 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.794124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.794176 (+ 52us) server_negotiation.cc:407] Connection header received
0620 23:08:08.794231 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.794234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.794247 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.794318 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.794581 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.795273 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.795962 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.796276 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.796366 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.798409 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.798418 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.798419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.798430 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.799863 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.800169 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.800171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.800171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.800219 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.800442 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.800442 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.800443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.800570 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.800634 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.800770 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.800882 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.188.124@KRBTEST.COM" created.
I20260620 23:08:08.801129 8446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.793896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:38533 (local address 127.0.0.1:37102)
0620 23:08:08.794109 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.794120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:08.794178 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:08.794326 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:08.794326 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:08.794459 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:08.794502 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.794503 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.795379 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.795380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.795867 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.795868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.796367 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.796367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.796430 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.796899 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:08.796903 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:08.798296 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:08.799938 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.799939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.799940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.800096 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.800288 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.800289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.800289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.800375 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.800633 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:08.800634 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:08.800695 (+ 61us) client_negotiation.cc:769] Sending connection context
0620 23:08:08.800748 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 23:08:08.800984 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":124}
I20260620 23:08:08.801776 8444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:08.802064 8446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.801982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52516)
0620 23:08:08.802011 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.802049 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260620 23:08:08.802208 8394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
I20260620 23:08:08.802644 8446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.802543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52532)
0620 23:08:08.802587 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.802617 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 23:08:08.802817 8444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:38513: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.188.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab.
Entry for principal kudu/127.17.188.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab.
I20260620 23:08:08.816108 18161 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRp9mku/build/release/bin/kudu
/tmp/dist-test-taskRp9mku/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.124:38513
--webserver_interface=127.17.188.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab
--principal=kudu/127.17.188.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.188.84:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.188.126:34165,127.17.188.125:38533,127.17.188.124:38513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:08:08.852036 8444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } attempt: 1
I20260620 23:08:08.852490 8446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.852366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52538)
0620 23:08:08.852426 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.852463 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 23:08:08.852823 8446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.852730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52548)
0620 23:08:08.852774 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.852798 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 23:08:08.852911 8444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:38513: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
I20260620 23:08:08.857939 8369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } attempt: 3
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
I20260620 23:08:08.864954 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.858280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37112 (local address 127.17.188.125:38533)
0620 23:08:08.858350 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.858352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.858373 (+ 21us) server_negotiation.cc:407] Connection header received
0620 23:08:08.858397 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.858397 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.858416 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.858470 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.858786 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.859276 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.860046 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.860271 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.860360 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.862484 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.862485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.862485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.862492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.863950 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.864331 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.864332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.864332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.864379 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.864558 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.864558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.864559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.864651 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.864700 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.864849 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.864901 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:08:08.865015 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.858190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:38533 (local address 127.0.0.1:37112)
0620 23:08:08.858249 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.858260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:08.858327 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:08.858486 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:08.858487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:08.858640 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:08.858706 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.858707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.859362 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.859363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.859948 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.859951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.860360 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.860361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.860438 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.860940 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:08.860945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:08.862402 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:08.864021 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.864022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.864022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.864262 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.864439 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.864440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.864440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.864502 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.864709 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:08.864710 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:08.864759 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 23:08:08.864833 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 23:08:08.864921 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 23:08:08.865382 8369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:08.865680 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.865578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52564)
0620 23:08:08.865628 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.865658 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 23:08:08.865998 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.865922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52578)
0620 23:08:08.865960 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.865982 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 23:08:08.866101 8369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:38513: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
W20260620 23:08:08.896535 8451 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:08:08.896718 8451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:08.896735 8451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:08.896790 8451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 23:08:08.897223 8369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } attempt: 1
I20260620 23:08:08.897612 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.897507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52594)
0620 23:08:08.897559 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.897591 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 23:08:08.897972 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.897881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52606)
0620 23:08:08.897927 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.897956 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 23:08:08.898057 8369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:38513: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
W20260620 23:08:08.898233 8451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:08.898273 8451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:08.898288 8451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:08.898331 8451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:08.898358 8451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:08.899780 8451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.188.84:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.188.126:34165,127.17.188.125:38533,127.17.188.124:38513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/krb5kdc/kudu_127.17.188.124.keytab
--principal=kudu/127.17.188.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.188.124:38513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.188.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:08.900035 8451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:08.900266 8451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:08.904827 8456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:08.904961 8451 server_base.cc:1034] running on GCE node
W20260620 23:08:08.904922 8457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:08.904826 8459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:08.905258 8451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:08.905560 8451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:08.906699 8451 hybrid_clock.cc:648] HybridClock initialized: now 1781996888906677 us; error 44 us; skew 500 ppm
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:08.909004 8451 init.cc:377] Logged in from keytab as kudu/127.17.188.124@KRBTEST.COM (short username kudu)
I20260620 23:08:08.909489 8451 webserver.cc:460] Webserver started at http://127.17.188.124:42855/ using document root <none> and password file <none>
I20260620 23:08:08.909698 8451 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:08.909740 8451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:08.909874 8451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:08.910921 8451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/data/instance:
uuid: "9c87a0d7e2bd4d07846f15725d32dd60"
format_stamp: "Formatted at 2026-06-20 23:08:08 on dist-test-slave-zpss"
server_key: "816422d5bded95069cdd2c069b2eaf76"
server_key_iv: "1190b8144a1c99c606a8c5157755f27e"
server_key_version: "encryptionkey@0"
I20260620 23:08:08.911270 8451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/wal/instance:
uuid: "9c87a0d7e2bd4d07846f15725d32dd60"
format_stamp: "Formatted at 2026-06-20 23:08:08 on dist-test-slave-zpss"
server_key: "816422d5bded95069cdd2c069b2eaf76"
server_key_iv: "1190b8144a1c99c606a8c5157755f27e"
server_key_version: "encryptionkey@0"
I20260620 23:08:08.912523 8451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:08.913178 8466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:08.913383 8451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:08.913478 8451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/data,/tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/wal
uuid: "9c87a0d7e2bd4d07846f15725d32dd60"
format_stamp: "Formatted at 2026-06-20 23:08:08 on dist-test-slave-zpss"
server_key: "816422d5bded95069cdd2c069b2eaf76"
server_key_iv: "1190b8144a1c99c606a8c5157755f27e"
server_key_version: "encryptionkey@0"
I20260620 23:08:08.913565 8451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:08.919597 8451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:08.921044 8444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } attempt: 2
I20260620 23:08:08.921406 8446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.921270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52612)
0620 23:08:08.921345 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.921378 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 23:08:08.921404 8451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:08.921550 8451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:08.921813 8446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.921677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52614)
0620 23:08:08.921729 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.921766 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 23:08:08.921969 8444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.188.124:38513: Network error: Client connection negotiation failed: client connection to 127.17.188.124:38513: connect: Connection refused (error 111)
I20260620 23:08:08.925257 8451 rpc_server.cc:307] RPC server started. Bound to: 127.17.188.124:38513
I20260620 23:08:08.925284 8518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.188.124:38513 every 8 connection(s)
I20260620 23:08:08.925669 8451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/data/info.pb
I20260620 23:08:08.926442 8519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:08.927806 8519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:08.930347 18161 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRp9mku/build/release/bin/kudu as pid 8451
I20260620 23:08:08.930472 18161 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRp9mku/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996599092328-18161-0/minicluster-data/master-2/wal/instance
I20260620 23:08:08.930652 18161 external_mini_cluster.cc:1428] Setting key ab4e08ff97c7bf2cb6f7062cb104855c
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
I20260620 23:08:08.937685 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.928715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.17.188.126:34165)
0620 23:08:08.928772 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.928775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.928815 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:08:08.929461 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.929462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.929474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.929534 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.930108 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.930646 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.932096 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.932378 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.932511 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.934738 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.934740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.934740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.934748 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.936023 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.936545 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.936546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.936546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.936596 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.936870 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.936871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.936871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.936997 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.937062 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.937473 (+ 411us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.937559 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:08:08.937809 8521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.928469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.126:34165 (local address 127.0.0.1:49556)
0620 23:08:08.928641 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.928727 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:08.929363 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:08.929596 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:08.929600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:08.929824 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:08.930010 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.930013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.930740 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.930741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.931992 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.931995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.932521 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.932523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.932673 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.933176 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:08.933186 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:08.934647 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:08.936122 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.936128 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.936130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.936453 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.936664 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.936665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.936665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.936790 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.937080 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:08.937087 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:08.937268 (+ 181us) client_negotiation.cc:769] Sending connection context
0620 23:08:08.937431 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 23:08:08.937592 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260620 23:08:08.938305 8519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:08.940490 8523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.930987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.17.188.124:38513)
0620 23:08:08.931135 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.931139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.931191 (+ 52us) server_negotiation.cc:407] Connection header received
0620 23:08:08.931258 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.931259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.931274 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.931339 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.932056 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.932711 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.934384 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.934659 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.934745 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.937390 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.937397 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.937397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.937403 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.939374 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.939740 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.939743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.939744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.939804 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.939975 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.939976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.939976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.940126 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.940269 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.940309 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.940386 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.126@KRBTEST.COM
I20260620 23:08:08.945196 8521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.938572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.125:38533 (local address 127.0.0.1:37118)
0620 23:08:08.938609 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.938618 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:08.938652 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:08.938753 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:08.938753 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:08.938937 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:08.938973 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.938975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.939599 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.939600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.940295 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.940296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.940912 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.940912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.940955 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.941390 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:08.941395 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:08.942522 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:08.944219 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.944220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.944221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.944391 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.944592 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.944593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.944593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.944658 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.944912 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:08.944912 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:08.944952 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 23:08:08.945021 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:08:08.945107 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 23:08:08.945211 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.938615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37118 (local address 127.17.188.125:38533)
0620 23:08:08.938659 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.938661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.938677 (+ 16us) server_negotiation.cc:407] Connection header received
0620 23:08:08.938697 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.938697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.938705 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.938748 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.939044 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.939519 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.940374 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.940695 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.940763 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.942611 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.942612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.942613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.942621 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.944129 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.944466 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.944467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.944468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.944526 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.944733 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.944734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.944734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.944845 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.944919 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.945053 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.945118 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:08:08.945551 8519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:08.948232 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.941280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.17.188.126:34165)
0620 23:08:08.941339 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.941341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.941406 (+ 65us) server_negotiation.cc:407] Connection header received
0620 23:08:08.941475 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.941476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.941485 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.941551 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.941981 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.942489 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.943316 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.943518 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.943603 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.945699 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.945701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.945701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.945707 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.947088 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.947457 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.947458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.947459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.947517 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.947724 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.947725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.947725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.947841 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.948026 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.948064 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.948137 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.124@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
I20260620 23:08:08.952608 8523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.946095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.17.188.124:38513)
0620 23:08:08.946398 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.946399 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.946406 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:08.946422 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.946423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.946432 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.946507 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.946721 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.947197 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.948054 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.948393 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.948475 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.950123 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.950124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.950124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.950131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.951468 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.951756 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.951758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.951758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.951815 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.952025 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.952026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.952026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.952177 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.952240 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.952401 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.952521 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":285}
I20260620 23:08:08.952659 8521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.946009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52626)
0620 23:08:08.946142 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.946151 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:08.946180 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:08.946510 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:08.946510 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:08.946626 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:08.946647 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.946648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.947269 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.947270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.947976 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.947978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.948488 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.948488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.948552 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.948940 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:08.948945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:08.950068 (+ 1123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:08.951539 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.951539 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.951540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.951689 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.951892 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.951893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.951893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.951958 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.952244 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:08.952245 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:08.952307 (+ 62us) client_negotiation.cc:769] Sending connection context
0620 23:08:08.952383 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:08:08.952479 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.188.125@KRBTEST.COM
I20260620 23:08:08.954298 8519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60: Bootstrap starting.
I20260620 23:08:08.955106 8519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:08.955389 8519 log.cc:824] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:08.956084 8519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60: No bootstrap required, opened a new log
I20260620 23:08:08.957234 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.948943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.17.188.125:38533)
0620 23:08:08.948974 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.948994 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.949458 (+ 464us) server_negotiation.cc:407] Connection header received
0620 23:08:08.949526 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.949526 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.949535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.949577 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.950047 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.950720 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.951430 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.951751 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.951820 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.954476 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.954478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.954478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.954485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.956021 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.956436 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.956437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.956438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.956500 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.956729 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.956730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.956731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.956838 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.957022 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.957059 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.957140 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:08:08.957919 8519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } }
I20260620 23:08:08.958101 8519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c87a0d7e2bd4d07846f15725d32dd60, State: Initialized, Role: FOLLOWER
I20260620 23:08:08.958225 8519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } }
I20260620 23:08:08.958469 8525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:08.958529 8519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:08.958554 8525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:08.958855 18161 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260620 23:08:08.960394 8536 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:08.960503 8536 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:08.959954 8343 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ffd9c9aa318547db964bfbbbb5a37434"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49568
I20260620 23:08:08.960644 8451 master_runner.cc:186] Error getting master registration for 127.17.188.126:34165: OK, instance_id { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" instance_seqno: 1781996888632684 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:08:08.962661 8451 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:08:08.973148 8369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } attempt: 2
Jun 20 23:08:08 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.126@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
I20260620 23:08:08.981076 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.973435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52638)
0620 23:08:08.973495 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:08.973521 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:08.973586 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:08.973724 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:08.973725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:08.973915 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:08.973964 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.973966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.974854 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.974856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.975713 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:08.975716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.976163 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.976164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:08.976222 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.976734 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:08.976740 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:08.978363 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:08.979928 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.979929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.979930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.980149 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.980345 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:08.980346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:08.980346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:08.980406 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:08.980732 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:08.980734 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:08.980779 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 23:08:08.980874 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 23:08:08.980981 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 23:08:08.981076 8523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.973479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.17.188.124:38513)
0620 23:08:08.973536 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.973537 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.973567 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:08:08.973645 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.973645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.973653 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.973700 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.974051 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.974768 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.975797 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.976082 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.976148 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.978447 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.978448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.978449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.978455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.979825 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.980211 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.980212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.980213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.980266 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.980455 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.980456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.980456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:08.980666 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:08.980732 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:08.980898 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 23:08:08.980981 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:08.982478 8369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434: Bootstrap starting.
I20260620 23:08:08.983223 8369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:08.983528 8369 log.cc:824] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:08.984373 8369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434: No bootstrap required, opened a new log
I20260620 23:08:08.985780 8369 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } }
I20260620 23:08:08.985924 8369 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd9c9aa318547db964bfbbbb5a37434, State: Initialized, Role: FOLLOWER
I20260620 23:08:08.985996 8369 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } }
I20260620 23:08:08.986313 8369 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:08.986279 8539 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:08.986370 8539 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:08.986563 8343 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 23:08:08.986649 8343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:08.987520 8343 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } }
I20260620 23:08:08.987794 8343 leader_election.cc:288] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [CANDIDATE]: Term 1 election: Requested vote from peers 71efd678cbcd4c63a1599d322b322ca1 (127.17.188.125:38533), 9c87a0d7e2bd4d07846f15725d32dd60 (127.17.188.124:38513)
I20260620 23:08:08.988651 8493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ffd9c9aa318547db964bfbbbb5a37434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9c87a0d7e2bd4d07846f15725d32dd60"
I20260620 23:08:08.988781 8493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:08.989671 8493 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd9c9aa318547db964bfbbbb5a37434 in term 1.
W20260620 23:08:08.989705 8550 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:08.989760 8550 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:08.989889 8319 leader_election.cc:302] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c87a0d7e2bd4d07846f15725d32dd60, ffd9c9aa318547db964bfbbbb5a37434; no voters:
I20260620 23:08:08.990051 8539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:08.990203 8539 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [term 1 LEADER]: Becoming Leader. State: Replica: ffd9c9aa318547db964bfbbbb5a37434, State: Running, Role: LEADER
I20260620 23:08:08.990288 8539 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } }
I20260620 23:08:08.990830 8557 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffd9c9aa318547db964bfbbbb5a37434. Latest consensus state: current_term: 1 leader_uuid: "ffd9c9aa318547db964bfbbbb5a37434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:08.991024 8557 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:08.992107 8561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:08.990742 8301 master_runner.cc:186] Error getting master registration for 127.17.188.125:38533: OK, instance_id { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" instance_seqno: 1781996888779384 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:08:08.992959 8301 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:08:08.993369 8561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:09.001055 8523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.990322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.17.188.124:38513)
0620 23:08:08.990352 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.990354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.993153 (+ 2799us) server_negotiation.cc:407] Connection header received
0620 23:08:08.993250 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.993251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.993260 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.993288 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.993986 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.994725 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.995757 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.996043 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.996080 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:08.997697 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:08.997699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:08.997699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:08.997705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:08.999593 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:08.999977 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:08.999978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:08.999978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:09.000029 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:09.000228 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:09.000229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:09.000229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:09.000364 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:09.000433 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:09.000626 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 23:08:09.000960 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:08:09.003818 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.990017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.17.188.125:38533)
0620 23:08:08.990072 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.990074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:08.992239 (+ 2165us) server_negotiation.cc:407] Connection header received
0620 23:08:08.992707 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:08.992708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:08.992720 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:08.992752 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:08.995655 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.996371 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.997981 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:08.998353 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:08.998409 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:09.000421 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:09.000422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:09.000423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:09.000440 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:09.002207 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:09.002772 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:09.002773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:09.002774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:09.002844 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:09.003124 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:09.003125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:09.003125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:09.003271 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:09.003340 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:09.003646 (+ 306us) server_negotiation.cc:299] Negotiation successful
0620 23:08:09.003725 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:09.005769 8493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [term 1 FOLLOWER]: Refusing update from remote peer ffd9c9aa318547db964bfbbbb5a37434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:09.006150 8557 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:09.007519 8525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffd9c9aa318547db964bfbbbb5a37434. Latest consensus state: current_term: 1 leader_uuid: "ffd9c9aa318547db964bfbbbb5a37434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:09.007601 8525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:09.008800 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:08.990689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.17.188.126:34165)
0620 23:08:08.990725 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:08.990727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.001277 (+ 10550us) server_negotiation.cc:407] Connection header received
0620 23:08:09.001396 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.001397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.001407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.001434 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.002143 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:09.002866 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:09.003986 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:09.004287 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:09.004325 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:09.005803 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:09.005804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:09.005804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:09.005811 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:09.007469 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:09.007835 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:09.007836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:09.007836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:09.007887 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:09.008068 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:09.008069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:09.008069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:09.008197 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:09.008302 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:09.008644 (+ 342us) server_negotiation.cc:299] Negotiation successful
0620 23:08:09.008715 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:08:09.009543 8562 mvcc.cc:205] Tried to move back new op lower bound from 7299059257364459520 to 7299059257304838144. Current Snapshot: MvccSnapshot[applied={T|T < 7299059257364459520}]
I20260620 23:08:09.009624 8539 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffd9c9aa318547db964bfbbbb5a37434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:09.009704 8539 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:09.009984 8525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffd9c9aa318547db964bfbbbb5a37434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:09.010064 8525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:09.010411 8563 mvcc.cc:205] Tried to move back new op lower bound from 7299059257364459520 to 7299059257304838144. Current Snapshot: MvccSnapshot[applied={T|T < 7299059257364459520}]
I20260620 23:08:09.013067 8444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } attempt: 3
I20260620 23:08:09.017915 8561 catalog_manager.cc:1347] Generated new cluster ID: 8986e18f2dc84b9f9f3581b67288135d
I20260620 23:08:09.017961 8561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:08:09 dist-test-slave-zpss krb5kdc[8288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996888, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.188.125@KRBTEST.COM for kudu/127.17.188.124@KRBTEST.COM
I20260620 23:08:09.022542 8446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:09.013299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.188.124:38513 (local address 127.0.0.1:52654)
0620 23:08:09.013394 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:09.013404 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:09.013439 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:09.013912 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:09.013912 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:09.014059 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:09.014096 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:09.014098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:09.015046 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:09.015047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:09.015873 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:09.015876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:09.016432 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:09.016433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:09.016498 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:09.016924 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:09.016929 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:09.018990 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:09.021373 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:09.021374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:09.021375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:09.021573 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:09.021806 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:09.021807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:09.021807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:09.021872 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:09.022170 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:09.022171 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:09.022226 (+ 55us) client_negotiation.cc:769] Sending connection context
0620 23:08:09.022285 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 23:08:09.022424 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260620 23:08:09.023375 8523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:09.013528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.17.188.124:38513)
0620 23:08:09.013568 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.013582 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.013590 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:09.013608 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.013609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.013618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.013667 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.014188 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:09.014891 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:09.016019 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:09.016319 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:09.016374 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:09.019090 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:09.019091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:09.019092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:09.019098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:09.021218 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:09.021661 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:09.021662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:09.021662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:09.021728 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:09.021943 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:09.021944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:09.021944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:09.022071 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:09.022125 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:09.023210 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0620 23:08:09.023289 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:08:09.024793 8444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1: Bootstrap starting.
I20260620 23:08:09.025516 8444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:09.025820 8444 log.cc:824] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:09.026712 8444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1: No bootstrap required, opened a new log
I20260620 23:08:09.028733 8444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } }
I20260620 23:08:09.028873 8444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71efd678cbcd4c63a1599d322b322ca1, State: Initialized, Role: FOLLOWER
I20260620 23:08:09.028954 8444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } }
I20260620 23:08:09.029254 8444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:09.029309 8418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ffd9c9aa318547db964bfbbbb5a37434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "71efd678cbcd4c63a1599d322b322ca1"
I20260620 23:08:09.029435 8418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:09.030256 8418 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd9c9aa318547db964bfbbbb5a37434 in term 1.
I20260620 23:08:09.030624 8417 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [term 1 FOLLOWER]: Refusing update from remote peer ffd9c9aa318547db964bfbbbb5a37434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:09.030879 8539 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:09.030932 8568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:09.031021 8568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:09.032716 8568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffd9c9aa318547db964bfbbbb5a37434. Latest consensus state: current_term: 1 leader_uuid: "ffd9c9aa318547db964bfbbbb5a37434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:09.032809 8568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:09.032855 8568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffd9c9aa318547db964bfbbbb5a37434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:09.032914 8568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:09.035962 8580 mvcc.cc:205] Tried to move back new op lower bound from 7299059257364459520 to 7299059257304838144. Current Snapshot: MvccSnapshot[applied={T|T < 7299059257364459520}]
I20260620 23:08:09.037214 8539 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffd9c9aa318547db964bfbbbb5a37434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd9c9aa318547db964bfbbbb5a37434" member_type: VOTER last_known_addr { host: "127.17.188.126" port: 34165 } } peers { permanent_uuid: "71efd678cbcd4c63a1599d322b322ca1" member_type: VOTER last_known_addr { host: "127.17.188.125" port: 38533 } } peers { permanent_uuid: "9c87a0d7e2bd4d07846f15725d32dd60" member_type: VOTER last_known_addr { host: "127.17.188.124" port: 38513 } } }
I20260620 23:08:09.037900 8539 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:09.038197 8561 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:09.039412 8561 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:09.041868 8579 catalog_manager.cc:1259] Loaded cluster ID: 8986e18f2dc84b9f9f3581b67288135d
I20260620 23:08:09.041957 8579 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1: loading cluster ID for follower catalog manager: success
I20260620 23:08:09.043205 8579 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1: acquiring CA information for follower catalog manager: success
W20260620 23:08:09.043377 8579 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260620 23:08:09.053416 8561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffd9c9aa318547db964bfbbbb5a37434: Generated new TSK 0
I20260620 23:08:09.053673 8561 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:08:09.075906 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.074347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.17.188.125:38533)
0620 23:08:09.074390 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.074392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.075268 (+ 876us) server_negotiation.cc:407] Connection header received
0620 23:08:09.075372 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.075373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.075385 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.075415 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.075827 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37156: BlockingRecv error: recv got EOF from 127.0.0.1:37156 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:08:09.076653 8523 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.074687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.17.188.124:38513)
0620 23:08:09.074724 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.074725 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.076126 (+ 1401us) server_negotiation.cc:407] Connection header received
0620 23:08:09.076271 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.076273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.076283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.076311 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.076596 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52658: BlockingRecv error: recv got EOF from 127.0.0.1:52658 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:08:09.077476 8372 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.074990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49578 (local address 127.17.188.126:34165)
0620 23:08:09.075027 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.075029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.076848 (+ 1819us) server_negotiation.cc:407] Connection header received
0620 23:08:09.077019 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.077021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.077032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.077061 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.077420 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49578: BlockingRecv error: recv got EOF from 127.0.0.1:49578 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:08:09.079681 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.078973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.17.188.125:38533)
0620 23:08:09.079006 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.079007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.079210 (+ 203us) server_negotiation.cc:407] Connection header received
0620 23:08:09.079308 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.079309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.079328 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.079356 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.079639 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37172: BlockingRecv error: recv got EOF from 127.0.0.1:37172 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 23:08:09.082671 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.081951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.17.188.125:38533)
0620 23:08:09.081981 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.081983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.082205 (+ 222us) server_negotiation.cc:407] Connection header received
0620 23:08:09.082288 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.082289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.082298 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.082324 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.082633 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37176: BlockingRecv error: recv got EOF from 127.0.0.1:37176 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:08:09.086441 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.085718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.17.188.125:38533)
0620 23:08:09.085749 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.085751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.085971 (+ 220us) server_negotiation.cc:407] Connection header received
0620 23:08:09.086061 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.086062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.086072 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.086099 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.086404 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37182: BlockingRecv error: recv got EOF from 127.0.0.1:37182 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:08:09.095602 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.094737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.17.188.125:38533)
0620 23:08:09.094775 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.094777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.095082 (+ 305us) server_negotiation.cc:407] Connection header received
0620 23:08:09.095193 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.095194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.095204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.095233 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.095559 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37196: BlockingRecv error: recv got EOF from 127.0.0.1:37196 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:08:09.105624 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.104736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37212 (local address 127.17.188.125:38533)
0620 23:08:09.104770 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.104772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.105018 (+ 246us) server_negotiation.cc:407] Connection header received
0620 23:08:09.105118 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.105119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.105129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.105174 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.105555 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37212: BlockingRecv error: recv got EOF from 127.0.0.1:37212 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 23:08:09.123757 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.122890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37228 (local address 127.17.188.125:38533)
0620 23:08:09.122930 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.122932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.123206 (+ 274us) server_negotiation.cc:407] Connection header received
0620 23:08:09.123306 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.123307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.123318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.123350 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.123702 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37228: BlockingRecv error: recv got EOF from 127.0.0.1:37228 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:08:09.158015 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.157160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37244 (local address 127.17.188.125:38533)
0620 23:08:09.157207 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.157209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.157360 (+ 151us) server_negotiation.cc:407] Connection header received
0620 23:08:09.157433 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.157434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.157446 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.157523 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.157954 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37244: BlockingRecv error: recv got EOF from 127.0.0.1:37244 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 23:08:09.223960 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.223168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37256 (local address 127.17.188.125:38533)
0620 23:08:09.223249 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.223251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.223354 (+ 103us) server_negotiation.cc:407] Connection header received
0620 23:08:09.223444 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.223445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.223457 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.223532 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.223895 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37256: BlockingRecv error: recv got EOF from 127.0.0.1:37256 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260620 23:08:09.354094 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.353159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37266 (local address 127.17.188.125:38533)
0620 23:08:09.353227 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.353229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.353348 (+ 119us) server_negotiation.cc:407] Connection header received
0620 23:08:09.353579 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.353580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.353593 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.353651 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.354032 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37266: BlockingRecv error: recv got EOF from 127.0.0.1:37266 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 23:08:09.611937 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:09.611293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37278 (local address 127.17.188.125:38533)
0620 23:08:09.611366 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:09.611369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:09.611419 (+ 50us) server_negotiation.cc:407] Connection header received
0620 23:08:09.611484 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:09.611485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:09.611497 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:09.611568 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:09.611886 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37278: BlockingRecv error: recv got EOF from 127.0.0.1:37278 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:08:09.960983 8536 catalog_manager.cc:1259] Loaded cluster ID: 8986e18f2dc84b9f9f3581b67288135d
I20260620 23:08:09.961053 8536 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60: loading cluster ID for follower catalog manager: success
I20260620 23:08:09.961838 8536 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60: acquiring CA information for follower catalog manager: success
I20260620 23:08:09.962172 8536 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c87a0d7e2bd4d07846f15725d32dd60: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:08:10.044003 8579 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 71efd678cbcd4c63a1599d322b322ca1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:08:10.127590 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:10.125150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37282 (local address 127.17.188.125:38533)
0620 23:08:10.125300 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:10.125302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:10.125362 (+ 60us) server_negotiation.cc:407] Connection header received
0620 23:08:10.125471 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:10.125471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:10.125517 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:10.125590 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:10.125723 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.126191 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.126891 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.127071 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.127117 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:10.127248 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:10.127359 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:10.127427 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:10.127461 (+ 34us) server_negotiation.cc:299] Negotiation successful
0620 23:08:10.127477 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260620 23:08:10.131883 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:10.129371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52668 (local address 127.17.188.124:38513)
0620 23:08:10.129541 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:10.129543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:10.129551 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:10.129595 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:10.129595 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:10.129655 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:10.129738 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:10.129854 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.130371 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.131145 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.131357 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.131426 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:10.131575 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:10.131672 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:10.131755 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:10.131775 (+ 20us) server_negotiation.cc:299] Negotiation successful
0620 23:08:10.131795 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260620 23:08:10.136323 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:10.133407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.17.188.126:34165)
0620 23:08:10.133532 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:10.133535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:10.133544 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:10.133587 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:10.133588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:10.133633 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:10.133716 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:10.133884 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.134610 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.135373 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.135607 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.135685 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:10.135892 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:10.135992 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:10.136047 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:10.136192 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 23:08:10.136219 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":54,"threads_started":1}
I20260620 23:08:10.136365 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:10.133749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.17.188.124:38513)
0620 23:08:10.133815 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:10.133825 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:10.133940 (+ 115us) server_negotiation.cc:407] Connection header received
0620 23:08:10.134000 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:10.134001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:10.134009 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:10.134056 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:10.134145 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.134596 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.135684 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.135888 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.135975 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:10.136106 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:10.136139 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:10.136183 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:10.136268 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 23:08:10.136293 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 23:08:10.136405 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:10.133357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.17.188.125:38533)
0620 23:08:10.133401 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:10.133403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:10.133628 (+ 225us) server_negotiation.cc:407] Connection header received
0620 23:08:10.133713 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:10.133713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:10.133723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:10.133771 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:10.133909 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.134580 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.135661 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:10.135927 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:10.135983 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:10.136149 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:10.136182 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:10.136230 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:10.136316 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 23:08:10.136337 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:08:10.141191 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 8301
I20260620 23:08:10.146510 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 8376
I20260620 23:08:10.151114 18161 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRp9mku/build/release/bin/kudu with pid 8451
2026-06-20T23:08:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2772 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5536 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291067 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS